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PREF ACE 


This paper describes the theory of a bibliometric analysis, outlines the limitations of a citation 
index to papers and journals, and presents selected results of a pilot project in bibliometrics 
involving an evaluation of the research papers and journals monitored by the Institute for Scientific 
Information. An overview of the above topics is contained in an earlier publication, An Indicator of 
Excellence in Canadian Science: Summary Report, Catalogue No. 88-507E. 


Science and technology indicators may be defined as statistics which measure quantifiable aspects 
of the creation, dissemination and application of science and technology. As indicators, they should 
help to describe the science and technology system, enabling better understanding of its structure, of 
the impact of policies and programs on it, and the impact of science and technology on society and the 
economy. 


An Indicator of Excellence in Canadian Science is one of a series of background papers on science 
and technology indicators published by Statistics Canada. The purpose of the series is to describe the 
theoretical development, limitations and application of various statistics suggested as indicators of 
science and technology. 


Current indicators of Canada's scientific and technological activities include: 


- expenditures on research and development ; 

- federal government scientific activities; 

- personnel working in science and technology; 

. Canadian research output (citations); 

. Canadian patented inventions; 

- international payments and receipts for technology; 
. trade in selected commodities. 


Statistical tabulations of the indicators will be released in Science and Technology Indicators, 
Catalogue No. 88-201, an annual summary; Industrial Research and Development Statistics, Catalogue 
No. 88-202 (annual); Resources for Research and Development in Canada, Catalogue No. 88-203 (annual); 
Federal Scientific Activities, Catalogue No. 88-204E (annual); and in a monthly service bulletin, 
Science Statistics, Catalogue No. 88-001. 


A list of the background papers is included at the end of this publication. These papers represent 
the opinions of the authors and do not necessarily represent those of Statistics Canada. Comments are 
invited and should be addressed to Humphrey Stead of the Science and Technology Statistics Division. 


This paper has been prepared by James MacAulay of the Information Analysis Group and edited by 
Karen Walker and Elizabeth MacKenzie. 


Martin B. Wilk, 
Chief Statistician of Canada. 
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INTRODUCT IGN 


This paper reports on a bibliometric study of the literature of science and technology. The purpose 
of the study was to develop a family of indicators to measure the results of research activities, and 
to evaluate their contribution to the science community. Such an undertaking was bound to be contro- 
versial because the evaluation of scientific literature has traditionally been left to the internal 
processes of judgment by one's peers. The methodology developed and implemented in the study however, 
uses publication and citation data, artifacts of the peer judgment process. 


While the pilot study described here was carried out in 1979 and 1980, there has been a long delay 
in publishing the results since they caused considerable controversy. The intervening time has been 
used to undertake a series of evaluations. The delay has been advantageous: time has proven that the 
objective procedure described here selected precisely the paper (among millions) which initiated a 
revolution in research areas throughout biochemistry and medicine. 


For several reasons existing classifications of the system of journals are unsatisfactory (e.g., 
subjective selection); a by-product of this study is the first systematic evaluation and clustering of 
journals and papers representing the whole of science. The structure of the journal system is reported 
only as it refers to the classification of journals, but additional avenues have been opened into the 
sociology and dynamics of science and technology as a system of knowledge.[1] 


The typical statistical distributions in this field are very skewed, and therefore, familiar des- 
criptive measures, such as averages that are based on central tendency assumptions, would have been 
misleading. Technical measures have been kept to a minimum, and as far as possible, a sequence of the 
same measures is repeated for all research areas. Appendices IV and VI have been organized with 
cross-references to assist the reader to access the results from any starting point such as a parti- 
cular journal or area of interest. 


Chapter 1 describes the theoretical aspects of a bibliometric analysis. The difficulties in using 
available data are examined and the publications collection of the Institute for Scientific Informa- 
tion (ISI) is assessed; the final section of Chapter 1 outlines an "analysis of relevance" which forms 
the basis of our study. Chapter 2 presents the results of a classification of the ISI journals 
followed by a presentation of descriptive indicators of the papers within these journals. Chapter 3 
focuses on the evaluative assessment of international performances in various research activities, and 
concludes with a specific examination of Canada's contribution to the literature of science. The 
results of the study are summarized in Chapter 4. 


See References at end of text. 
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Chapter 1 


THE DEVELOPMENT OF A BIBLIOMETRIC STUDY 


Introduction 


For any analysis of science a key issue is how to define the subject of study. Within the science 
community, one can identify a variety of different sources of information including professional 
scientists and associations circulating old science or generating new science. This community is sup- 
ported by institutional resources and facilities, technically competent individuals, as well as libra- 
ries filled with reference literature. It is reasonable to suggest that a common focus for all members 
of this community is scientific and technical knowledge. And, it follows, that knowledge, or more 
correctly, intellectual property must be one of its most crucial possessions: the element that ani- 
mates and motivates all its members. Scientific publication functions as a communication system within 
the community such that contributions to scientific knowledge can be circulated and acknowledged, and 
thus attain to their value. 


In Appendix I the relationship between science and the literature of science is discussed. The 
formal use of published accounts of research to establish priority claims to knowledge, i.e., intel- 
lectual property, is put into historical context. We see that the research paper has become the chief 
tool for making priority claims in intellectual property and for having such claims evaluated by 
peers.[2] 


Judgment by one's scientific peers would seem to be a process that takes place in at least two 
stages. Peers initially appear in the somewhat elite form of gatekeepers: evaluating proposals, 
reviewing accomplishments for tenure and other awards, and editorial refereeing of papers. The second 
stage of peer evaluation is definitely more democratic. Publication is essentially a knowledge claim 
which calls for recognition by the entire scientific community for the author's priority in the disco- 
very of new knowledge.[3] One of the most common responses is the citation. The citation is of special 
interest because, while it can be used in both pejorative and substantive ways, it is at all times a 
formal and objective record by the scientific community at large which recognizes the knowledge claims 
of others, and which represents an evaluation of these claims. Publication and citation therefore, can 
be taken to represent declarations of achievement and recognition respectively. 


The discipline of bibliometrics makes use of the fact that knowledge must be documented in order to 
be formally communicated, making it possible to develop measures of selected characteristics of the 
records and/or documents generated in this process. The purpose of a bibliometric analysis is to 
derive measurable variables from a system of documentation that are characteristic of the scientific 
knowledge represented therein, and of the research community in which it is to be circulated. The pur- 
pose of the analysis here is to derive a family of indicators that are descriptive of trends in the 
direction and levels of activity within science, and evaluative with respect to the quality and rele- 
vance of that output; and more specifically, to derive indicators of excellence in Canadian science. 


Such bibliometric indicators provide useful information for the consideration of science policy 
issues at both national and organizational levels, and for testing hypotheses which arise from the 
social studies of science. Applications of the latter type stress an analysis of the structure and 
dynamics of research areas. We are concerned here with policy-relevant indicators which have a dif fe- 
rent emphasis - the identification of output trends and the quality of papers at national, sectoral 
and organizational levels. 


The limitations of a bibliometric study including the possible sources of error which may affect 
such an analysis will now be reviewed. 
Limitations 


The first bibliometric exercises were intended as an aid to biblical scholarship. The "Concordantae 
Morales", by tradition ascribed to Anthony of Padua (1195-1231), was a concordance of words, with 
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cited passages of their occurrence, from Scripture. In the thirteenth century, a Dominican Cardinal 
repeated the exercise using an older version of the Bible, and employing 500 of his brothers in the 
task.[4] 


This tradition of scholarship was revived in the word frequency studies of G.K. Zipf in this 
century, and more recently in the field of type-token mathematics and psycholinguistics.[5] The 
Zipfian distribution of word frequencies has also been observed by A.J. Lotka with respect to the pro- 
ductivity of authors, and by S.C. Bradford in his studies of the distribution of papers on a subject 
among journals.[6] 


These studies demonstrate that, under given conditions of use, the elements of an information 
source will exhibit high degrees of relatedness and interdependence, with relatively few of the ele- 
ments satisfying most of the demand in a highly ordered and predictable fashion. The point is most 
obvious in studies of vocabulary use in texts where grammatical rules and subject requirements deter- 
mine that words will not be selected independently; we shall return to this idea in developing a meth- 
odology for analysing "relevance" relations among source elements in the following section. 


However, much of the current work in bibliometrics does not draw upon these and other developments 
in the field of information science (see Appendix II). As a result, one can find a number of biblio- 
metric studies based upon the following unwarranted assumptions: information is objective and measur- 
able; information can be measured as a flow from independent source elements to users; and the infor- 
mation source and its user population can be arbitrarily selected. The possible errors to which these 
assumptions can lead are divided into three types. 


Errors derived from failing to take into account intrinsic characteristics of science and scientific 
literature. Examples of this type of error arise if one does not acknowledge the radical variations 
among disciplines in the production, citation behaviour, rate of self-citation as well as, variations 
in authorship characteristics. 


One of the most fundamental characteristics of science and its literature is the very high degree of 
organization. Each discipline has its own history and institutions and, when inquiring within one's 
own field, one is on familiar ground. Disciplinary characteristics will strongly influence biblio- 
metric exercises such as publication or citation counting. Within a subject literature which is well 
enough defined to be studied over a long time span, the average number of publications per author 
appears to remain constant, despite radical changes in the cognitive structure of the field and its 
population of investigators.[7] On the other hand, from one discipline to another, this productivity 
ratio varies radically. In turn, the number of citations and co-citations which might be expected from 
papers will vary among disciplines. 


Editorial practices also vary considerably among disciplines. For example, page limitations will 
affect the number of references in a paper, and costs will presumably keep down the number of papers. 
Some fields maintain high rejection rates, which should elevate the quality of accepted publications, 
thus increasing the likelihood that published papers will be cited. 


The norms of publishing and referencing behaviour must also reflect variations in both content and 
style as well. For example, referencing in the development of mathematical theorems will be sparse 
compared with that in review articles; and no doubt the frequency of self citations and negative refe- 
rences is a matter of normative constraint in at least some fields.[8] Team work prevails in many 
research areas, particularly those involving large-scale apparatus or remote locations. In turn, such 
factors as the numbers of co-authors and the ordering of co-authors on a paper, which is clearly a 
matter of style, must be taken under consideration. 


Errors introduced by inherent limitations of particular methodologies. Much of the bibliometrics field 
has developed around techniques which contain unrealistic assumptions about scientific communication 
as a flow of information from source to user, representing these by publications and citations. These 
studies also reflect tendencies which prefer technique to theory, frequently using subjectively 
derived and grossly aggregated groups of journals to represent research areas. Journal groups that are 
too heterogeneous and ill defined do not allow the development of measures that might otherwise over- 
come the two other types of errors. To be sure, the great majority of bibliometric studies have indeed 
been conducted on a small scale, beginning with a bibliography of a subjectively confined research 
area, or generating a bibliography by building up collections of co-cited papers.[9] Many of the pro- 
blems associated with the information source or data base (see below) are not apparent until these 
procedures are subsequently and uncritically adopted for general use. 


Attempts at validating bibliometric results by correlating them with the more elitist forms of peer 
evaluation have proven quite encouraging as shown by the following examples of bibliometric exer- 
cises. Assuming that university departments represent roughly comparable fields of research, the 
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Roose-Anderson and Cartter assessments of graduate programs have in turn been correlated with simple 
bibliometric indicators of the same departments.[10] In his review of some 24 correlations of simple 
publication and citation indicators with evaluations involving the Roose-Anderson, Cartter and other 
non-bibliometric forms of peer evaluation, Francis Narin observed that, with respect to publications: 
"Peer evaluations have been found to correlate positively with citation rates, with the more highly 
cited publications generally more highly rated by the scientists' peers". Moreover, with respect to 
the evaluation of investigators: "In almost all cases, the bibliometric measures of eminence ascribed 
to groups of scientists will correlate reasonably well in the range of 0.5 to 0.8, with other eminence 
rankings". Finally, with respect to departments, "Most studies of this type show correlations in the 
0.7 to 0.9 range".[11] The bibliometric measures used in these correlations represent simple citation 
counts with no attempt to overcome the sources of error discussed in this chapter. 


With the strong correlations in favour of institutional-scale bibliometric indicators, Narin went on 
to develop, following Eugene Garfield, a weighted average impact factor for journals, based on the 
citations generated per reference made.[12] He then developed indicators of the departmental output at 
various universities, weighting it by the impact factor of journals, and found stronger correlations 
with Roose-Anderson than were found simply using publication output.[13] These encouraging results 
still require subjective groups of journals, and of course, they are not specific with respect to the 
quality of papers. 


On the other hand, the work of Henry Small on co-citation analysis addressed itself specifically to 
the quality and clustering of papers. By commonly citing two or more source papers, an author not only 
recognizes their separate knowledge claims, but groups or indexes them together in a single account. 
Thus, in identifying frequently co-cited pairs of papers or authors, Dr. Small and others developed a 
technique which, given a threshold condition, could cluster elements in a small homogeneous source and 
at least draw attention to the more important linkages relative to others in a very small information 
source.[14] However, as an impact factor, it is still subject to the biases which follow from the 
possible errors listed, including self and prestige citing; and when normalized, the co-citation link- 
age is not directional.[15] This means that, for example, by citing oneself and Einstein frequently, 
one could easily appear to a co-citationist to be quite important and part of an inner circle in an 
unwarranted way. Moreover, co-citation measures must be carefully normalized before they can be gene- 
ralized from one field to another. 


Nevertheless, co-citation did shift attention from the essentially false notion that one can measure 
information flow, toward the more fruitful approach of measuring relations among documents. Myer 
Kessler had contributed by developing a method of bibliographic coupling which groups documents with 
similar bibliographies, as opposed to grouping those with similar citation files.[16] An influential 
paper by Professor de Solla Price also drew attention to the clustering of "networks" of cited papers, 
although in his calculations, he was concerned with network dynamics rather than with its internal 
structure, or an evaluation of its components.[17] 


Errors derived from the character of the information source or data base. These include geographical, 
linguistic and disciplinary bias in the available sources of bibliometric data. Other factors to be 
taken into account include variations in editorial policy, on both a national and disciplinary basis, 
with respect. to rejection rates and lengths of articles. This error type will be discussed in detail 
as the adequacy of the ISI data base is a central factor to be considered in the design of the pilot 
study. 


The Institute for Scientific Information (ISI) provides valuable data for bibliometric studies. In 
1978, ISI monitored more than 4,000 journals and selected monographs drawn from all areas of the 
natural and social sciences for input into several information services including the Science Citation 
Index (SCI) and the Journal Citation Reports (JCR). The only available large-scale citation indices 
are in fact compiled by ISI. These listings are dictionary controlled, however, only for journals and 
papers within those journals monitored by ISI. That is, computer dictionaries of journal name 
variants exist only for the ISI subset of the cited literature. 


To assess the adequacy of the ISI coverage of research literature, we will use a comparison with an 
overall picture of the world research literature as provided by the acquisition tapes of the British 
National Lending Library. While the many abstracting services probably do make the most ambitious 
attempts of all bibliometric sources to provide complete coverage of subject literatures, they have 
not been merged into a single listing. Variations in abstracting style and content are inevitable, but 
they would not be critical for crude aggregations.[18] The ISI collection provides the only available 
bibliometric source for large-scale multi-disciplinary studies using citations. The main point of what 
follows is to indicate the nature and biases of that collection and to outline a methodology which 
overcomes those biases. 


See References at end of text. 
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The extant literature of science is vast. While the degree of redundancy in science publishing is 
not known, it seems reasonable to assume that most current research finds its way into the serial 
literature, which, in general, includes periodicals, regular proceedings and reports and excludes 
irregular reports, monographs, textbooks and the rest of science publishing. Several estimates of the 
number of periodicals have been suggested, many of them based on the "World List of Scientific Perio- 
dicals".[19] This publication unfortunately is cumulative, including inactive titles, and uncritical 
in that it includes newsletters and bulletins.[20] With the acquisition of computerized tapes of lite- 
rature titles by abstracting services and by some of the major science libraries in the world, it 
becomes possible to refine these lists, and to arrive at estimates of the currently available serial 
bale wesis 


Refined library acquisition lists probably yield the most realistic estimates of those journals 
freely circulating in the world science community, and, for bibliometric studies, it is reasonable to 
ignore titles which are not widely available. For example, there appear to be many Chinese titles 
which have not been made available, and reports on research in classified areas which are often inter- 
rupted indefinitely. 


The British National Lending Library attempts to acquire all available periodicals, 1i.e., publica- 
tions "intended to be continued indefinitely", in science and technology. This includes natural, 
physical and engineering sciences, and encompasses psychology, medicine and agriculture. It excludes 
social sciences except from some borderline journals. Mark Carpenter and Francis Narin obtained compu- 
ter tapes for the 1973 holdings of the British Library Lending Division (BLLD), which contained 
records of the 43,457 periodicals actually received, and of these, some 24,801 were thought to be 
actual research periodicals in the fields of science outlined above.[21] The number should not be 
directly compared with the earlier estimate by K.P. Barr, since he also examined records of periodi- 
cals on order as well as received. (Barr had found 22,619 research titles being received in 1965, with 
a further 3,616 on order. Experience at BLLD was that 60% of titles on order are eventually received. ) 
Taking this into account, both estimates converge on a figure of 25,000 to 26,000 research periodicals 
for the mid-1960s. The high mortality rate among titles was seldom taken into account in the past, and 
it introduces an element of calm into the picture of the information explosion. While the list is 
changing in content, the doubling rate must now be set at about 60 years.[22] 


Computerized library lists also include the addresses of the journal publishers and therefore, the 
geographical and disciplinary coverage of various collections can be compared. In fact, Carpenter and 
Narin have taken a preliminary step toward a comparison of this sort, which throws considerable light 
on the adequacy of coverage of the world's journal literature by ISI. They assigned the 24,801 re- 
search periodicals at BLLD to nine general subject fields. These periodicals were cross-referenced by 
country of publication to give a distribution of world research journals as shown on Table 1.1. A sub- 
jective allocation of titles was used for the limited purpose of description and comparison in the 
absence of any complete evaluation. This BLLD distribution is in fact the first available representa- 
tion of world research journals with enough satisfactory detail concerning its construction to be con- 
sidered reasonably accurate and complete. 


Table 1.1 presents an interesting picture: over 80% of all journals come from North America and 
Europe and over 40% from English-language areas. The United States is predominant, contributing at 
least 20% of all journals in most areas. The skewness toward a mainstream in journal publication turns 
out to be even more pronounced in the publication of papers and citations. This journal distribution 
is not a suitable proxy indicator for research activity because of the international character of many 
journals, and one must remember that the strongest journals are very cosmopolitan in their authorship. 
However, 1t 1s clear that the communication infrastructure for world research is heavily dependent on 
the resources of the mature research communities of the United States and Western Europe. It would be 
interesting to identify the degree of this dependence, but the data (1.e., source of cited paper 
rather than source of journal) are not at hand for all countries. 


Turning to the ISI journals included in the Science Citation Index (SCI) for 1975, and supplement - 
ing their subjective allocation procedure with a look at the range of references in the journals, 
Carpenter and Narin allocated 2,240 journals to the nine fields on Table 1.1. That is, they found that 
SCl was covering about 9% of the 24,801 BLLD journals in these fields. As we shall see, from our own 
more refined analysis of their relevance, the distribution of journals is highly skewed toward the 
great importance of a small percentage of them, and therefore 9% is probably more than would be needed 
overall to represent and analyse research areas in an objective way, with the important proviso that 
the higher quality journals were in fact being selected and that coverage was equitable. In fact, it 
1s estimated that 85% of the 5 million references thus monitored in a year by ISI do go to the small 
percentage of journals covered in the SCI subset. 


See References at end of text. 


TABLE 1.1 Distribution of World Research Journals by Country and Field 
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Biology Chemistry Physics 
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Bo) 7is5) lee 
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science technology 
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79 7 26 184 833 
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37 11 19 154 464 
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ee 


Source: Pritish Library Lending Division; M. Carpenter, F. Narin, "The Subject Composition of the World's Scientific Journals", Scientometrics, 2:1 
(1980), p. 53; from the BLLD acquisition tapes for 1973. 


16 


While the overall extent of SCI coverage would seem to be adequate, the fact is that not all impor- 
tant journals have been selected, and the distribution among fields and among countries is very 
uneven, more than enough to seriously distort most international comparisons based on aggregate publi- 
cation and citation counts. Table 1.2 provides a picture of this distribution comparing the SCI jour- 
nals to those in the BLLD listing. 


TABLE 1.2 Comparative Distribution of BLLD and SCI Journals(1) 


Clinical Biomedical Biology Chemistry Physics Earth and Psychology Mathematics |Engineering | Total 
medicine research space and 
sc lence technology 
eee |e eae | oe eee. 

BLLD | SCI BLLD | SCI BLLD |SCI BLLD BLLD BLLD 

a 
United States ........ 947 234] 269 99 920 daz 242 55 191 40 By) 44 180 56 132 SiG See 198} 4,987! 870 
United Kingdom ....... 333 76 174 37 602 58 173 12) 145 15 229 7 60 15 ili 11] 1,346 96) 3,136) 344 
IESSSAING BoOoKODOodOOS 228 10 149 11 320 3 95 a 160 14| 367 4 4 1 74 Yi 784 AT 2igulee We) 
East and West Germany 399 76 94 27 278 24 84 21 89 13| 214 10 46 Z 77 TA 702 34] 1,983 221 
JAPAN varofelorevoralstelerersvalelers 352 19 82 11 292 8 53 8 52 4 130 4 1; 2 36 Le Saal Td) We 20 71 
PIMC]! cleleicicleieicleseieie eres 296 51 65 9| 210 16 ya] 5 44 8 158 4 27 2 41 51 410 A A278 sat 
DENED Sieicieicicreiclcrsicieie viele 1,962 177 579 80/2,228 Po) 294 60] 289 PACH i ieee ely 31 115 Wi 406 20}2,664 EDEL Al 538 
Total oc. c. ceccccccce 1,417 643) 1,209 274| 4,850 294; 965 186; 970} 123)/2,982 114| 445 91 853 87| 8,010 428 | 24,801/ 2,240 
Row percentage ....... 18.2 eva) 2s5) (v4) Was gina Bie) e155) 5.9 Sei) 2.0 Dieu 1.8 4.1 3.4 FoF! S205) Aen | (OUCH OCG 

per cent 
United! States eir.s-sicl 21.0 3604) 22.2 DG | ae) SB Ui ZDie 1 AVS Wai) Sasa) Sa 38.6) 40.4 | 61.5) 15.5 SZ) N69) | AG. 5) 2.0 S88 
United Kingdom ....... 7.4 11.8] 14.1 13.5) 12.4 TAN Whee) PAN ASH) a? Wiel WER eee 16.5] 9.0 12.3) 16.8.) (22.4 12.6] 15.4 
WESESSIRG oocodancscadn 5.0 Eley] W265} 4.0} 6.6 oleh) ols DoT AGao WN Thee 12:01 Dre) | ee) oa 8.7 Zed Fe8 4.0 8.8 B05) 
East and West Germany 8.8 11.8 7-8 928 Dro, BiZ | Bies Vilas) “Sahl allblates| 7a Beal) 10> Zee FeO) 25g i8e8 US) BO! 9.9 
JERE oHopooucabeconte 7.8 3.0 6.8 4.0) 6.0 Zell Bie 4.3 5.4 Seo) beet 5D eS) 22 462 4.5 6.4 2.6 6.1 Bai? 
[ane metelsleleleloleinieterersrs:s 6.6 oy) Dial Find |) ASS De) ae Pei TERMI (5c) Byep) 5335) 6.1 2.2| 4.8 Dien youl 4.0 wile Daa 
DENCE Foie cic o's 6 cro ice ols (MESS) || PAGE| Bhilosi Tea Io\\ TSS ACN | PANE SAO a7 52.2) 298) 29556) 46.2 Diie2 |) 208 14.3] 48.8 PSST WEIS) NZ, 39.2) 24.0 
Total ..cccccccceccsce 100.0 |100.0/100.0 | 100.0/100.0 |100.0|100.0 | 100.0/100.0/100.0| 100.0 | 100.0/100.0 | 100.0|100.0 | 100.0/100.0 | 100.0) 100.0) 100.0 

1) BLLD Journals - 1973 data; SCI Journals - 1975 data. ae 

Source: M. Carpenter, International Science Indicators, Computer Horizons, Inc. (Cherry Hill, New Jersey, 1979), Table I-9. 


The results present us with some problems. Looking across the rows in the top of Table 1.2, i.e., 
the distribution of journals among fields, and for example, recalling a rather large number of engi- 
neering and technology journals in the BLLD data base (32%), we find here a much smaller share for the 
SCI (19%). On the other hand, the SCI coverage of biomedical research is notably larger than BLLD 
coverage (5% of BLLD, 12% of SCI), and this difference is repeated in clinical medicine (18% and 29%, 
respectively), chemistry (4% and 8%) and psychology (2% and 4%). Indeed this concentration of SCI 
journals in clinical medicine has been a bias since the late 1950s when the collection was initiated 
by Dr. Garfield to construct a genetic citation index. 


A different kind of problem arises in mathematics where, while the presentation receives the same 
emphasis in the two data bases it is unevenly distributed among countries, as shown on the lower half 
of Table 1.2. Here the two major geographical biases in SCI are very evident: a serious under-repre- 
sentation of mathematics journals from the Soviet Union, which has 9% of BLLD mathematics and only 2% 
of SCI mathematics, and an over-representation of American journals in the area. For comparisons of 
international performance we note that language differences result in certain countries (particularly 
the U.S.S.R.) being infrequently cited by western authors but that authors in these countries, in most 
cases, reserve their best papers for those journals that are recognized internationally. A clustering 
analysis (the procedure developed for this study) will tend to identify and isolate these linguisti- 
cally specialized journals, drawing into the mainstream only those which are cosmopolitan and actually 
used by the world community. This means then that while representation can be small, it must be rea- 
sonably accurate, and include all journals likely to have significant influence. Where the numbers are 
small, as in mathematics, missing a few key foreign-language journals, or a few key specialized jour- 
nals will undermine the most ingenious procedures for obtaining comprehensive clusters of that field. 


From our own point of view, we are concerned with an evaluation of Canadian research, in terms of 
its relevance in the world community. Therefore, the SCI overemphasis of English-language journals 
suggests that Canadian research may be over-represented in terms of crude publication and citation 
counts. Data for Canadian journals were not available for Table 1.2, but an impression can be formed 
of the situation for Canada from the following considerations. 


UE 


In the BLLD journal set for 1973, of the 24,801 publications, Canada published 587 journals, or 
2.4% of the total and there is no reason to assume that this share would change between 1973 and 
1977. But in an examination of 1977 ISI data, Eugene Garfield reported 46 Canadian journals, or only 
1.8% of the SCI set for that year. This was despite the very strong overall representation of Canadian 
authored articles in the SCI set of 3.9%. One of the reasons for this is significant here: Canadian 
researchers work in an environment which is in many ways associated with the largest research commu- 
nity in the world. This has had profound implications throughout our research infrastructure, and 
particularly for the development of independent research institutions in this country, including pro- 
fessional associations and journals. 


Without analysing this point in detail, the full extent of Canadian cosmopolitanism in research 
publishing was described empirically by Garfield.[23] He observed 14,238 source articles in the 1973 
SCI journal set (including notes, editorials and reviews), which had indicated a Canadian location as 
their first listed address. As shown in Column 2 of Table 1.3, less than 30% of these were published 
in Canadian journals, while 50% were published in U.S. journals. He then looked at some of the cita- 
tion patterns to these articles over the period 1973-76. The percentages of Canadian papers that were 
cited ranged from 38% for the few that appeared in Danish journals to 78% for those in West German 
journals, with 66% of those placed in Canadian journals. More important, of the Canadian papers that 
were cited, the citation rate was usually higher if they had been published abroad. As shown in Column 
5, the rate was only 4.1 cites per cited Canadian paper published in Canada, but 6.5 for those pub- 
lished in the United States, and 7.2 in the Netherlands, where a number of major international pub- 
lishing houses are established. 


TABLE 1.3 Distribution of Canadian Authored Articles 


Canadian Allocation Share Average 
Place of papers(1) to coun- cited citations 
publication tries 1973-76 to%cited 
papers 
number per cent per cent 
Unmeedmatal ES iy. stesso elciace 6 stalsintchs, «is : 7,070 NS AY 69.9 G29 
CERECR 835 AS eaae lr SOOO rons 4,091 Zoe 66.0 4.1 
Ent.e Cig KIMGGOMin shes ie snl ctelers ole siete cour be 1,439 Vos 69.0 Dee 
Meter LMG a hey ei steieie «ine eo cinerea 679 4.8 TST Toad. 
iMiil Jl ANIC piel ate: ole whe sisle suelo afaleregeiexe  « 235 WG 7 Fe 4.7 
MGUM ET Manly amcretetstsie sic siete etocele cteys esac «6 199 1.4 78.4 ae 
ERTQIIGE) Wote's cbetele sievetere oleieis serieysie aie) e ees 107 0.8 aL eae) 
DETIM ATK aig s of6'e ste)», cole 0reielenssb eeceyelaiece » 6 6 106 0.7 Dileeth 6.1 
Die irate ata wo steicis ties ne ole erel tetera atone cere o1zZ Des 44.9 Le? 
VOU ALi eo a stateres eisis'e sles» oie int lew cjore a 14,258 100.0 64.9 Die 


(1) Includes articles, notes, editorials and reviews monitored in 1973. 
Source: £. Garfield, Current Contents, 34, August 1979. 


Noting the chosen language of the Canadian articles, Garfield also presented information which 
throws some additional light on the linguistic character of mainstream research, and the dilemma of 
francophone Canadian scientists. Less than 4% of Canadian articles were published in French, and of 
those only 39% were cited, a significantly lower rate than for English-language articles. In contrast, 
64% of the English-language articles were cited. Thus, we observe that Canadian scientists as a whole 
publish heavily in the English-language mainstream of SCI journals, and their work tends to be more 
heavily cited when they do this. 


See References at end of text. 
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Summary 


Several points have been made about the citation data base accumulated at ISI. To summarize the 
implications of these observations for our study, the following points are noted: 


- An objective and thorough evaluation of the ISI data base, in terms of journal representativeness 
and importance has not yet been carried out. Though it is needed, the evaluation would require ma- 
chine readable dictionaries to read the variants of cited journal names, where these journals are 
not covered by ISI. Since 1975, reasonably reliable computer dictionaries exist only for reading 
variant names of the ISI source journals. 


The existing dictionaries permit the observation that the overall range of journals in the SCI data 
base (9% of the BLLD collection) is more than adequales Since 85% of the references monitored in a 
year are from SCI monitored journals. 


A disciplinary and geographical distribution of SCI and BLLD journals demonstrates significant 
biases in the SCI representation toward biomedicine and toward U.S. representation, and to a lesser 
extent, toward coverage of English-language journals. 


Canadian journals are not over-represented, but the journals in which Canadians publish are over- 
represented (the latter point does not hold for Canadian social science). As the basis for a study 
of Canadian research in the natural sciences, the range of representation appears adequate. The two 
biases in the distribution tend to cancel each other: since fields which are under-represented in 
SCI such as engineering, biology, earth and space sciences, and mathematics are, in fact, the fields 
in which there is an over-representation of U.S. and English-language coverage. The biases are more 
than enough to render simplistic bibliometric measures artifactual, but the methodology presented in 
Appendices II and III was designed to obviate an uneven disciplinary coverage. 


. Direct international comparisons of publication or citation counts would not be warranted, given 
these biases, except in cases where an effective control could be introduced and trends established 
(for example comparing incremental growth in country shares of papers in journal sets which are held 
constant and represent the same general areas of research). 


Variations in the coverage of fields, variations in publication frequencies among research areas, 
and expected variations in the communication character of specialties at different stages of matu- 
rity, require that bibliometric indicators be developed only with respect to objective representa- 
tions of those fields and research areas. 


To conclude; these considerations together with other characteristics of the ISI data base (such as 
error levels and variants of authors and addresses), impose several constraints on procedures for the 
development of bibliometric indicators: 


an evaluation and clustering of journals is required for the development of reasonable time series 
for national productivity in particular disciplines; 


an initial (journal-driven) strategy, implementing some analysis of the ISI representation of 
research journals, followed by an evaluation of papers identified by a journal/volume/issue/page/ 
year code, would seem to be called for in order to minimize the error levels in author and article 
names; and 


additional analysis is required, based upon a methodology such as that outlined in the following 
section, in order to use the citation data in either a general or an evaluative way. 


Comput ing Relevance Among Source Elements 


In the previous sections we have seen where some of the difficulties lie in using the data avail- 
able. Bibliometric records must be used with caution to measure achievement and peer recognition, 
particularly with respect to heterogeneous and large-scale collections of documents. Biases are pre- 
sent in both the data base and subject fields. This section first outlines a theoretical framework for 
evaluating the relationships (i.e., relative influence) among elements of an information source as a 
means of overcoming these difficulties and second, defines mathematically a measure of relative in- 
fluence, termed "relevance", used to evaluate these relationships and thus the quality of contribu- 
tions to research. Appendices II and III will provide more specific information about the procedures 
for implementing these ideas. 


NIE 


As discussed in "Limitations" bibliometric techniques have achieved some success in evaluating the 
quality of research at an institutional level within given fields of literature, and in developing 
maps of research specialties as perceived by the research community. But precision in both the evalua- 
tive task and in the structural analysis of source documents requires some stronger theoretical base, 
particularly for large, heterogeneous information sources. In mentioning the Lotka-Bradford-Zipf 
distributions, we have, in fact, already referred to the traditional theory of information science 
from which our theoretical base derives: science is an inquiring system.[24] Investigators, operating 
within the normative, cognitive and technical constraints of scholarly communities, inquire for infor- 
mation relevant to their dynamic state of learning. At the most primitive level, nature might be seen 
as the source of information. But investigators cannot confront nature directly nor at random; and, 
taking into account the cumulative characteristic of knowledge and the complex of symbolic systems and 
social relations which make this possible, the immediate source of information becomes the research 
community itself. We have noted above, that of all the information systems which may be active in a 
research community, special importance is attached to that particular, formal one which in fact pro- 
vides a system for publishing and acknowledging knowledge claims. 


Inquiry and learning are intellectually costly processes. The limitations of short-term memory and 
human information processing within social groups impose serious constraints on researchers. They must 
accept a variety of classification and coding strategies, for example, selecting from a small core of 
favoured source elements (the "Matthew effect" of elitism in science which turns up in the Bradford- 
Zipf distributions), rather than inquiring at random over the whole source. A source of scientific 
information, such as the community of researchers itself, turns out to be functionally divided into 
research areas, and the relative importance of documents and individuals within these areas in turn is 
radically skewed. That is, under given conditions of use, an information source naturally partitions 
into classes and orders itself with respect to the importance of class elements. 


Inquiry and learning are also time-dependent, and thus what will be pertinent to a user of an 
information source is very much a function of what the investigator already knows, and hence, where he 
or she is presently inquiring for pertinent information. Accordingly, users trace out quite specific 
directional pathways among the elements of an information source, ordering it and pointing to its most 
important elements. By formalizing these relations mathematically, we can find the most important 
source elements, and classify the source. Depending upon the level of specificity required, source 
elements can be identified as either papers, or authors, journals or institutions. Our implementation 
involves cited papers and cited journals. 


Given a file of citing papers for each cited paper, we can derive two probability measures: (1) the 
marginal propensity of authors to inquire from one paper to another, represented by W. Goffman's tran- 
sition probability,[25] and (2) the relative frequency with which each of these ordered pairs of cited 
papers is chosen from among all ordered pairs of cited papers in the citation index. These two mea- 
sures are the basis of the term we call "relevance". 


The transition probability measures the conditional probability that given an author makes refe- 
rence to any given paper, he will, in the same article, make reference to any other specific paper in 
the whole source of cited papers. This means the two papers are associated together in terms of 
intellectual distance (i.e., a relation of influence exists between the two cited papers). Moreover, 
after a few months' worth of accumulated citations, a direction will have been established showing the 
greater probability of citing investigators to move toward particular papers. While the transition 
probability reflects the relation of influence between papers by measuring the propensity of investi- 
gators to inquire across any given linkage, it suppresses the relative importance of the various link- 
ages (i.e., pairs of cited papers). Sensitivity to this factor is introduced by measuring, in a compa- 
rative way, the contribution of each linkage to all possible linkages within the citation index. 


Relevance therefore defines a set of mathematical relations among papers, the properties of which 
are most important, and permit us to order these pairs into chains; partition them into exclusive 
classes; obtain a measure of the relevance of any cited paper to all other cited papers (the sum of 
the relevance of each of the linkages to a paper from all other papers);[26] and obtain a measure of 
the relevance of any class of cited papers (the sum of the relevance of all papers in that class). 


The transition probability helps to overcome several of the biassing factors mentioned previously. 
For instance, in the case of self and prestige citing, repeated self-cites would have no biassing 
effect on this measure, and the directionality distils from prestige citing a correct measure of the 
influence of Einstein upon oneself, and not the other way around. Similarly, for a data base that 
over-represents some part of a system of papers, transition probabilities reflect the actual use of 
what is there, the way that inquiring users make use of actual and always incomplete collections. 
Moreover, the intrinsic variations in citation rates from one research area to another do not bias 
this measure; i.e., other things being equal, a directed path of a mathematics cluster of papers will 
compute in a similar way for one in chemistry, though the production or citation rate may differ by an 
order of magnitude. 


See References at end of text. 
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The transition probability, however, has two major defects. Firstly, it was designed for small 
homogeneous data bases accessed by a given single set of users. This means in defining clusters of 
related papers a single threshold could be set on the marginal relevance required by a linkage to 
augment the cluster. Secondly, within a cluster of papers, it submerges the relative importance of the 
various linkages. That is, in the same research specialty, a transition probability computed from five 
co-citations on an initial citation file of 30 will be the same as a much less travelled and corres- 
pondingly less important (peripheral) linkage of 1 in 6. Thus, to overcome this latter problem, the 
relative importance of each linkage of cited papers was introduced. With respect to the other problem 
of variations in a large data base, an (incremental) approach was devised that allows for a computer 
algorithm to select a threshold of marginal relevance to be established which is appropriate for each 
cluster, or:researeh area. 


To summarize, in practice a large-scale application will encounter many imperfectly monitored spe- 
cialties, linguistically isolated journals, and emergent specialties. The user population, and indeed 
the data base, must be considered heterogeneous. Therefore, the marginal relevance of linkages in a 
chain of cited source elements requires a threshold condition that is sensitive to local conditions of 
inquiry (Appendix III). Given an algorithm which meets this condition, we can make use of the analy- 
tical power of Goffman's approach for sources that are very large and heterogeneous. The strategy 
consists of making use of the mathematical properties of a directed graph of relevance relations on 
the source. Technically, a chain of relevant linkages imposes a transitive ordering relation on the 
source, while elements which are interconnected by such chains (defining a symmetric relation) may be 
partitioned into mutually exclusive classes. An algorithm was devised (Appendix III) which computes 
the relevance of source elements and which grows clusters of elements by inputting marginal relevance 
incrementally, thus classifying source elements in terms of those which are interconnected by networks 
of relevance relations. 


Relevance relations reflect a collective evaluation of influence among papers in a scientific 
field. Perfunctory, prestige and self citing behaviours not withstanding, the citation indicates an 
acknowledgement of knowledge claims. However, because of variations in citing behaviour and because of 
the biases which result from the error types noted earlier, the citation cannot be quantified as a 
measure of information flow. Rather, it can be used to derive two probability measures which are not 
sensitive to these sources of error and no amount of self, prestige or perfunctory citing, nor bias in 
monitoring the source, will alter the relations of influence which are thus collectively imposed by 
the research community itself. By formalizing these relations, we can find the most important source 
elements and discover a working classification of the source. 


Chapter 2 


CLASSIFYING THE SYSTEM OF JOURNALS 


Methodology 


The relevance theory was applied to both the journals and the papers within the journals of the ISI 
collection. A computer algorithm was devised and used to first evaluate the relevance of all ordered 
pairs of the 4,000 ISI journals; and then to evaluate the relevance on all ordered pairs of papers 
published in those journals in the three selected years 1974, 1975 and 1976, based on their citation 
files for 1978. Separate procedures were used for the journals and papers since computational effi- 
ciency, critical to the problem of clustering algorithms (see Appendix III), is quite different in 
each case. 


In practical terms there is no limit to the possible size of a system of informational elements 
whose relevance relations are to be computed (essentially it can be resolved into a computational and 
sorting problem); but on the other hand, the clustering part of the evaluation is more severely cons- 
trained, and until a graph theoretic approach was found, they had been limited to very small informa- 
tion sources of about 100 elements or less.[27] The result of this was an excessive level of subjec- 
tive interference and selection. 


The algorithm developed for this study is optimal in terms of efficiency and, a standard mainframe 
with current enhancements can handle about 20,000 elements (a matrix of 4 x 108 cells), which is large 
relative to the world's journals, but still small in relation to all cited papers in a period of three 
years. Thus an initial journal analysis was required to classify groups of related journals which 
would then form the basis for classifying the papers that turn out to be relevant in a separate eva- 
luation of cited papers. The clustering part of the algorithm operates incrementally, reading in 
ordered pairs of source elements, from the most relevant linkage to the least relevant; and then 
forming clusters of interconnected elements in a systematic fashion. In the case of the journal analy- 
sis, when a marginal linkage failed to augment the accumulated relevance of a class of journals by 
more than 0.5%, the class was considered sufficiently complete or saturated. Such a class or intercon- 
nected network of source elements is called a research front, and in this case, a research front of 
journals. 


In the early 1970s, ISI devised a computer dictionary to read the variant names of those cited and 
citing journals monitored at the Institute. On the basis of this dictionary, a journal-to-journal 
citation index called the Journal Citation Reports, is produced. At the journal level of aggregation, 
a citation index becomes rather a blunt instrument of analysis, particularly with respect to multi- 
disciplinary journals. In fact, the analysis could be made much more specific but only the JCR was 
available for the pilot project. 


An analysis of the tapes of the 1978 JCR was carried out; relevance values were computed for some 
4.6 million ordered pairs of cited journals; they were rank ordered and then read into the clustering 
algorithm. The ordering displays a characteristically skewed form of a log-normal or Bradford-Zipf 
curve. The first research front saturated when the top 400 relevance pairs had been read in. Indeed, 
what one might consider to be the core of the system contains only 40,000 of a total 4.6 million 
ordered pairs or linkages of cited journals with non-zero relevance in the collection, but this small 
network accounts for about half of the system's relevance. After the first 56 research fronts were 
identified (i.e., had saturated), the remaining journals were increasingly peripheral to the system 
frequently consisting of unit classes with little influence on the rest. Thus, if we include only 
those fronts which exert some influence on at least one saturated front (i.e., only those journals 
found to be at least minimally influential), 85 fronts in the system were identified. They contain 
just 1,468 of the original 4,000 journals, but they account for 96% of the total relevance of all 
linkages in the system. 


The clustering algorithm is iterative, and can be re-run to find super-classes (classes of 
fronts). A class or group of related research fronts is called a research field. Thirty-six such 
fields were identified in the journal system representing rather general or broader areas of science. 
The journals classified in this way are listed alphabetically in Appendix VI. A journal class was 


See References at end of text. 
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called a research front following Price's terminology, and was denoted by a field: front sequence, as 
in R.F.i:j, where i = field and j = front (see Reference 17). Appendix IV provides a concordance of 
fronts into fields along with the results of the journal classification) exercise,, listings each 
research front as it was saturated in the analysis. The names suggested for each front are for the 
convenience of the reader and result from a scan of journal titles in each front. 


The Journal Collection 


The work on the system of journals is of interest because it is really the first time a structural 
analysis of any sort has been carried out on a set of documents representing, however, imperfectly, 
"the whole of science". The analysis of the citation data for 1977 from the 1978 JCR yields the 
following results. 


The top-ranking journals were in the following order: Nature, Science, Journal of Biological 
Chemistry, N.A.S. Proceedings, Biochimica et Biophysica Acta, Lancet, New England Journal of Medicine, 
Journal of Clinical Investigation, Annals of New York Academy of Science, and tenth, Journal of the 
American Chemical Society. Physical Review Letters and Royal Society of London, Proceedings, Series 
A also appeared very high in the ranking, numbers 29 and 48 respectively; and the top Canadian journal 
was Canadian Journal of Chemistry, journal number 54. 


Eighty-five clusters of journals (i.e., research fronts) that had at least a minimal degree of 
relevance/influence on the collection were identified. Fifty per cent of the total journal collec- 
tion's relevance is contained in the first 22 research fronts identified (96% is contained in the 85 
fronts). 


These 85 fronts contain 1,468 "relevant" journals of the original 4,000 considered and are identi- 
fied in the order of their influence on the system. The most influential research front, Core Bioche- 
mistry (R.F.1:1), as shown in Chart A, is a multi-disciplinary cluster of 12 journals related through 
their orientation to biochemistry. These journals contain papers in areas outside of biochemistry but 
it is their central influence in biochemistry which predominates. 


Chart A. Journals Comprising the Research Front of Core Biochemistry 


HS 


Relveicalis = 


oe 


1. NATURE, 1869, London, England, 14.4%, E. 

Ze spoClENCE e854 4 Washingt any, se gut o lame. 

3, JOURNAL OF BIOLOGICAL CHEMISTRY, 1905, Bethesda, Md., U.S.A., 12.0%, E. 
4. NATIONAL ACADEMY OF SCIENCES, 1915, Washington, D.C. 102670) Ee 

5. BIOCHIMICA ET BIOPHYSICA ACTA, 1947, Amsterdam, Netherlands, 10.3%, E. 
6. BIOCHEMICAL JOURNAL, 1906, Colchester, England, 8.2%, E. 


7. BIOCHEMICAL AND BIOPHYSICAL RESEARCH COMMUNICATIONS, 1959, New York, U.S.A., 7.0%, E. 


8. BIOCHEMISTRY (U.S.), 1962, Washington, D.C., 6.5%, E. 
9. ARCHIVES OF BIOCHEMISTRY AND BIOPHYSICS, 1942, New York, U.S.A., 5.6%, E. 


10. EUROPEAN JOURNAL OF BIOCHEMISTRY, 1967, Berlin, West Germany, 4.2%, E, F, G. 


11. JOURNAL OF MOLECULAR BIOLOGY, 1959, London, England and New York, U.S.A., 4.1%, E, F, G. 


12. FEBS LETTERS, 1968, Amsterdam, Netherlands, 4.0%, E, F, G. | 
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Within each research front, the journals are listed in order of their total relevance in the entire 
system. Thus, reading under R.F.1:1, we find the journal Nature, founded in 1869, published in London, 
England supplying 14.4% of the relevance of all journals in the front and published in English. Note 
that the five strongest journals in the whole system of journals form a "clique" at the centre of 
R.F.1:1. The influence of the 12 journals in R.F.1:1 extends throughout the system. On a directed 
graph representation of interfront relevance relations (a relevance digraph), this fact is evident in 
the large numbers of relevance pathways from all fields into the central core of the graph. 


This core group of fronts contains, in addition to R.F.1:1, the research fronts of Core Medicine 
(R.F.1:2), Core Chemistry (R.F.4:3), and Core Physics (R.F.3:4). Their influence extends throughout 
the entire journal collection and together they evidently represent the "central core of influence" of 
the whole collection. 


After this central core of research fronts was identified (i.e., sequentially saturated by the 
algorithm) the groups which follow first tend to represent the core areas in more generalized and then 
more specialized or linguistically isolated forms. Also, many of the less influential fronts frequen- 
tly consist of unit classes with little or no influence on the rest of the collection. These tend to 
represent imperfectly monitored research specialties, newly emergent specialties or linguistically 
isolated journals and are considered peripheral to the mainstream research fronts which are interna- 
tionally representative or cosmopolitan. 


The field which links the main clinical and biomedical research fronts together contains six 
closely related research fronts: Core Biochemistry (R.F.1:1), Core Medicine (R.F.1:2), Pharmacology 
and Clinical Chemistry (R.F.1:5), Cell Research (R.F.1:6), Gastroenterology, Surgery (R.F.1:7) and 
Immunology and Pediatrics (R.F.1:8). 


Core Chemistry (R.F.4:3), which saturated as the third front, is very strongly intra-connected, and 
therefore integrated, and formed a field by itself. It is, of course, central to the rest of the phy- 
sical sciences. The front is dominated by the Journal of the American Chemical Society and Journal of 
Chemical Physics, but the Canadian Journal of Chemistry is actually the sixth strongest journal in the 
world mainstream of chemistry publications. 


Summar y 


The collection would appear to be highly integrated with respect to all of science and technology 
as evidenced by the interclass linkages; this contradicts the prevailing view that these are separate 
knowledge systems. For example, though we have adopted the journal citation tool as an obviously blunt 
and preliminary instrument, our subsequent evaluation of the influence of papers refines, but nowhere 
contradicts, the general observation that the entire system is highly integrated. The research jour- 
nals of science and technology define a system of knowledge, and that system appears not to have a 
simple lattice or tree structure, built up from the most basic research areas to the most applied. 
Instead, system structure spirals outward from a central core, reproducing the mainstream classes in 
more specialized or linguistically isolated and peripheral forms. 


For several reasons, the journal evaluation results should be considerably more reliable than 
impact factors.[28] However, our purpose here is not to refine a classificatory system for science, 
but to implement a methodology which overcomes the general problems inherent in the development of 
bibliometric indicators and the limitations of the ISI data base. Thus, the minimal requirement of the 
journal analysis is to partition and order the system of journals into fronts in a non-subjective 
fashion, so that the subsequent analysis of papers and derived indicators can be made meaningful. 


Preliminary Descriptive Indicators 


As noted, the central purpose of the pilot study was to develop a method to analyse papers using 
the relevance measure as the chief "evaluative" indicator. However, given a classification of the sys- 
tem of journals into comparable homogeneous groups, as outlined in the previous section, it is easy to 
develop a group of descriptive indicators based upon a time series of papers and co-authorship 
counts. The actual number of publications for any country would not be meaningful by itself, but use- 
ful indicators can be developed: for example, by controlling the research fronts of journals over 
time, one can reasonably look at incremental changes in national shares, or, where a country is 
strongly represented in a research front, relevant trends in the productivity of particular institu- 
tions may become apparent. 


See References at end of text. 
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As previously noted, while Canadian journals are not over-represented in the data base, the jour- 
nals in which Canadians publish are undoubtedly over-represented in a global context. This is a good 
thing for the purpose of our analysis, but of course it requires caution in interpreting product ivity 
levels internationally. 


International Shares of the World's Scientific Papers 


Controlling for the introduction of new journals from 1974 to 1978 (and for increases in journal 
size by taking share values), Canada's share of the world's scientific papers fell slightly from 4.6% 
to 4.5%, as did shares for the United States: (45.3% to 43:5%), and’ Britain (9.8%tto 9.4%). Table: 2m 
presents the distribution and change in world share of papers by journal front for Canada, the United 
States and England, for those fronts in which the three countries at least minimally participated. 


Measuring national changes in production levels by the percentage change of 1978 production over 
1974 production, and taking only the 32 most influential journal fronts in the system, we may note 
that the number of Canadian papers increased by at least 5% in 14 research fronts, declined by at 
least 5% in 11 and remained within that range for seven others. This pattern was similar to that for 
the United States and England, with 15 and 14 significant gains respectively. (Since the numbers 
reflect national coverage at ISI, international data is reserved for the discussion of relevance below 
and national shares are summarized here.) It is unlike the pattern for Japan, Italy, West Germany and 
France, where gains showed up in 28, 27, 24 and 22 research fronts respectively (data not provided). 
This pattern in Western Europe is inconsistent with the argument that mature research communities like 
the United States and Britain are merely maintaining their world share. In fact, the data would seem 
to indicate an overall process of redistribution of research with gradually more activities being 
taken up outside the English-language mainstream. 


The numbers of papers from developing countries is still too small in particular fronts to test the 
notion of a redistribution statistically, but we can measure the overall dispersion of activity among 
countries by means of standardized gini and entropy measures. These measures were computed for 23 
fields of journals involving at least 20 countries, and in all cases but one, the dispersion of acti- 
vity increased among countries from 1974 to 1978 (again, we are summarizing a large volume of data 
which cannot practically be provided). The one exception was field 1 (including R.F.1:1, 1:2, 1:5, 
1:6, 1:7 and 1:8, representing the most influential research fronts in biochemistry and medicine). The 
number of countries participating in these fronts increased slightly from 113 to 117 in the period; 
and the concentration of activity among them, when standardized for comparison, also increased. But in 
all other cases, the opposite tendency was evident, indicating a slow process of redistribution of 
activity in the physical sciences away from the most concentrated regions of the United States and 
Britain. For the most part, it means a greater participation rate in Western Europe, Australia, 
Israel, South Africa, and to a lesser extent, some East European countries. There is as yet little 
Sign of the long-anticipated redistribution of particular research activities to Third World coun- 
tries. 


While these shifts in influence are undoubtedly a good sign for the health of the world research 
community, research in Canada, unlike that in the United States and Britain, is not yet fully develo- 
ped in all sectors and should still be expanding in terms of activity levels. 


Table 2.1 provides some details of Canadian activity, showing where increases in world share have 
taken place. As discussed earlier and evident from Appendix IV, many of the research fronts are very 
small or isolated linguistically and thus, relatively unimportant from our point of view. The range, 
as for the United States, is very broad, indicating that we are not a specialized research power. Of 
the 80 journal fronts in Table 2.1 Canada participated in 77 of them in 1978, publishing 100 or more 
papers in 41 of them. We increased in world share between 1974 and 1978 for 22 of the fronts and 
decreased for 19. In only three of the positive increments did we increase world share by more than 
1%: in Ecology (R.F.2:21, with journals in ecology, marine biology, entomology); in Clinical Pharmaco- 
logy, Geriatrics (R.F.2:43) and in Animal Biology (R.F.10:38 with journals in animal behaviour, zoo- 
logy, applied ecology). These all represent areas dominated by governmental and regulatory science. It 
is the kind of science normally associated with an early stage in the evolution of a country's 
research infrastructure: a period often rather unkindly but accurately referred to as its colonial 
period. 


Table 2.1 also shows that the Canadian research community experienced six significant declines in 
world share. These tended not to be in areas of colonial or regulatory science and the stewardship of 
resources, but unfortunately involve more fundamental areas. For example, our share of papers in 
Microbiology and Macromolecular Science (R.F.2:15) declined by two percentage points; this is an 
important group of journals in microbiology, genetics and polymer science. Another significant decline 
in world share was in Solid State Physics (R.F.3:10) including journals in micro-electronics and sur- 
face sciences. This represents the knowledge base for our developing micro-electronics industry, and 
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TABLE 2-1 World Shares of Papers by Journal Front for Selected Countries(1) 
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Represents a count of country addresses for all papers monitored in 1974 and 1978, respectively. 


Eng land 

Per cent 1974 Per cent 1978 Per cent 
world world world 
share share share 
Bye) 1,742 9.78 1,652 8.63 
62.61 3,376 14.56 4,067 15.03 
48 .90 ent tes hod 1,066 10.16 
60.00 561 6.20 636 Sipe) 
FEES 569 8.30 625 8.03 
EE) aS) 561 Meee 1,090 7.07 
48.67 453 7.65 384 9.54 
34.13 529 8.60 362 7682 
43.02 818 7.26 901 6.55 
62.50 138 Bist? 147 4.16 
41.98 280 71483 367 7.66 
68.47 305 9.36 289 1202 
43.92 479 11.05 473 8.67 
47.35 654 7 LET Uh 683 18.21 
32.01 323 3.55 461 4.64 
29.654 149 453, 175 5.26 
89.13 10 ATS: 1 1.09 
31.96 406 11.05 635 14.81 
36.24 113 12.83 267 20.24 
Lhe Hs 217 Date 646 at epee] 
30.93 thi 2.82 248 5618 
44.09 1,067 6.56 889 Bg fe 
16.95 1,496 16.87 inky? {22 
32.47 748 6.88 690 5.78 
50.03 403 5.01 421 4.70 
45511 788 Deke 757 4 Bz 
Zora 338 Ailvndae 423 12.07 
20.76 454 6.16 444 ASE: 
4.06 141 ie! 102 1.03 
Eire ys 267 7.00 295 WEE) 
60.88 51 Zuo 37 2.09 
43.22 507 8.17 467 Fe70: 
63.74 60 Dalit 55 5.56 
Tasers 227 4.46 197 5.49 
28.85 197, aeo> 294 8.45 
14.27 98 3.65 103 ee) 
pis hf 95 4.23 145 Sail 
3.96 51 rey 56 3.41 
bere 417 5.47 478 6.65 
19.78 957 dipeeed 732 10.26 
62.17 106 8.64 149 5.62 
54.44 oy 4.45 152 oop | 
67.38 51 5.65 69 Diet 
40.63 93 10.05 127 Vw 22 
55.86 5 6.76 7 6.31 
56.98 240 11225 270 10.31 
69.54 177 10.30 153 11 65 
17.82 44 DEB) 74 4.47 
BG Ve 392 6.71 276 4.67 
31.70 66 5.28 141 ei 
78.38 3 2.08 5 4.50 
44.89 64 11.09 57 Tice. 
79.09 20 8.30 29 4.56 
58.50 330 10.37 zt) 7.85 
INS) 37 157 6.95 212 9.46 
63.78 12 0.98 73 Ae54 
47.43 412 9.28 438 10.25 
67.03 5 3.09 S| 1.65 
41.71 30 10.07 39 ee) 
60.41 108 9.36 213 Cys 
59.69 15 3204 22 4.85 
1.24 2 0.16 4 0.23 
54.13 53 18.60 35 17.30 
70.77 50 1 bteiliz 28 10.77 
Os55 1 0.54 1 B53 
42.72 43 5456 46 4.82 
Deete 13 0.65 151 4.22 
6s. 10 4.13 lags 11.63 
54.65 17 12.06 16 9.30 
- - - 1 0.65 
8.97 16 16.16 18 23.68 
1S oe 44 40.00 80 50.63 
57.32 1 1.09 6 51 
7.80 - - 8 3.67 
52.86 - - 4 Patt 
87.99 - - - - 
95.30 - - - - 
14.58 16 32.00 16 b bo PO 
- - 23 10.75 


26 


is an area, as we shall presently see, in which Canadians contribute a high proportion of significant 
papers. A decline in activity here, as in microbiology, could turn out to be costly in several res- 
pects. The decline in Plant Biology (R.F.5:11), covering botany and cell physiology, was from an 
overly high level of activity in an area of science that, while important to agriculture, reflects the 
scientific preoccupations of our colonial past. With reference to Main Mathematics (R.F.6:17), we have 
noted that our participation level may be artificially high because of seriously incomplete coverage. 
Nevertheless, a loss of more than two points in world paper shares is not at all a good sign, and 
again, this occurs here in the face of a disproportionally hign Canadian share of the top papers (12%, 
see below). Another decrease in world share is in Applied Mathematics and Computer Science 
(R.F.14:47), involving 37 important journals in computer science, automation and control theory, nume- 
rical analysis, etc. Finally the sixth decline was also in an area of economic importance; this time 
involving government science to a larger degree and the result of a drop from a high level of acti- 
vity. This area, Secondary Veterinary Science (R.F.22:68) is represented by a small group of eight 
journals in animal research and veterinary medicine. Recent institutional development in this area by 
Agriculture Canada's Research Division may by now have reversed this trend. 


Table 2.2 shows some of the trends in levels of research activity for countries with at least 1% of 
papers in the three most strongly connected fronts (Core Biochemistry R.F.1:1, Core Medicine R.F.1:2, 
and Core Chemistry R.F.4:3). U.S. publications increased from 10,153 to 10,618 papers from 1974 to 
1978 (see Table 2.1) in the 12 key journals of Core Biochemistry, representing a drop in world share 
from 57.0% in 1974 to 55.5% in 1978. The absolute increase results from increases in the publication 
of papers by the journals themselves - this is normal for most journal classes. In Core Biochemistry 
the number of Canadian publications fell by 5.2% from 1974 resulting in a slight drop in our share of 
world activity. We maintained a share of about the same magnitude in the main medical group, although, 
in order to do this, the actual number of Canadian papers increased from 697 to 845, in line with a 
corresponding increase in the papers published in those journals. In the Core Chemistry research 
front, the Canadian volume dropped by 3% from 1,116 to 1,083 papers and our share of all papers in the 
front dropped slightly to about 7%. The large and expanding volume of activity in Japan and West 
Germany in this area is notable.[29] 


International Co-authorship Linkages 


Co-authorship among countries is another useful descriptive indicator which is readily available 
for controlled journal fronts. This analysis is based on co-authorships identified as pairs of 
addresses. 


Canada had 11,555 linkages of this kind in the address records of the 1,468 relevant journals for 
1978; 8,868 of these were combinations within Canada. That is to say, 23% of the co-authorships that 
involved at least one Canadian address, also involved a foreign address. In terms of this measure, 
Canadian research would seem to be slightly more cosmopolitan in the period from 1974 to 1978, since 
22% of Canadian co-authorships involved a foreign author in the earlier year. 


It is often remarked that when a Canadian researcher chooses professional societies, schools for 
graduate training, recruiting and collaboration, or journals in which to publish, the United States 
will usually turn out to be the first choice. However, there are signs that Canadian researchers are 
increasing their involvement with other countries. Table 2.5 provides an indication of our 
international involvement through co-authored articles. 


While the distribution of Canadian co-authorship linkages among countries remained about the same 
over the period (the 13 countries listed accounted for 92% of all our international linkages in both 
1974 and 1978), within that distribution, there was a diminishing emphasis on the United States and 
considerably more activity with researchers from France, Australia, Japan, Sweden, Italy and Israel. 
Links to East European countries are still sporadic, 12 with Poland for example, and eight with 
Hungary; and similarly for Third World countries, with the exception of India, where some traditional 
ties have survived. 


Pursuing this indicator into more controlled conditions, we can briefly observe the international 
linkages in Core Biochemistry (R.F.1:1), all of which are well-known international journals involving 
a number of specialties. The international co-authorship matrix for R.F.1:1 (not provided) is again a 
symmetric matrix, each column representing the frequencies of co-authorship. Without reproducing this 
very large matrix, we note that a well-known centre-periphery phenomenon is evident therein, with 
access to and control of the main international journals obviously a key ingredient of centre-status. 
The column for the United States indicates a cosmopolitan range of interaction with 51 other countries 
for 1974, with considerable depth in a number of the entries. Twenty-five per cent of these linkages 
were with British addresses, 11.0% Canadian, 8.0% West German, 7.0% Italian, 5.5% Dutch, 5.2% French, 
4.6% Swedish, 4.4% Japanese and 4.3% Israeli. Major East European linkages included 14 (or 1.3%) with 
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TABLE 2.2 Changes in World Shares of Papers for Three Journal Fronts 


Country 1974 19 


78 


per cent 

Core biochemistry (R.F.1:1) 

UipIbee Ces tcl Sas cusses evehaceuers ela rerslio ser sence, sfene-0: sxcueseis eiel-ssus;euece ie siclotetetelolsterelstekeiars slekelislexeteisleveicusieroterereleltielers 57.0 2 
RGN caséasnwoessa sect Bisheleletatevete SuSdasSoudouadodar atetolsteteleierate crate aNershevatetoielo\eicteisiatelereieistelsieyet 9.8 
WES: (GMT Saddontodcoodconocdduno stele cov cheleletatoovstaterelevoielals cietslelel siete teleloveleleieletel dlclels ice’ sl cisiicle.s cleles Sc) 
PEO G0 GON ORROD SOULS OOD OUALODOTIOCLOT OOOO OG Sb00d6.60000 0nctad00 605006 aioli sieNeleVal oetatelecnstele S060 4.0 
DADAM elelcielvisic « <faveisre « SOKO GOO OODUOOUOO sfohe/oletstellelfelis si avsie\e) ayaieieve/e) efeye eis) s/e/e\e « e/'s).e)o!ssicheiele eles) slsis els) siete Syeitl 
ESINESE) BooooeCmocOGOd oOo FOOHCAO Do Gas DODO ODOODBGO0 OOOUOd HADODD OOOO Siolelalcraletelalarcisicrsieieleielstere 3.6 

SWE CEM avatele ole alepetcle.e aero < sicpeke.alerersxeteis aol icliovevekeleleietsreserater ore oisiereveteye)alsveusterere’ era) sreve eielealstetenelccereheteielsnenote AG) 
Netherlands ..... ELoverotelovonsislelatetereretarsietenelelcherere teres atelicleeveterefclcteletetstonctenctercietereyeverelerevareravaleveloteiatelcrcrer store Tae 
SINAC MMevalerare she. cette aiele sievevoherelerets oteveveielereretere e's OOF CDOIOADON O10 CLO Ow OOOO OLODCOOIG eretehetatenersieteiereleieteverete Wie 
tsi Vautatatal ofatelcraletehara efe/ets alelersierererciete rete = Biers teiore c\euster ete’ c aia ahetetoialaohsiels) eistelelicvsvelsleucliclsiietajaveledsi tel ote sete clei led 
GWilile ZOLA Ge stoves byereye «6 Sheree ~ Gtetovelaleuste ole ehcheleteleiatete’ sielel slelevelcloneys) cele thre ele eVeTanal oheuassel(ev chel'evevenetele evs eters la 
MUBIPAIMGAY GasodooSdudccpdocbcdDoCUoONOOOU CGO GOaoO eYelereycvererate afore otehstorsloroionsteleletelerslenccetccerclalalcrstore 1.6 
Sealants ooaccedoosaoosodooocoouooun forotoneLenelalefeterctereralavelelcletckarel oreo siicteietereratctererereraoneterclieveieterelcvarerele lo? 
BOGHU SaooosngcétouKosa0GNC0 afetelerehorsloteteya(olelcisistelsioters svetaevelsts sonoodco5os eloteletelelsieVeleretate ceeesce ° 0.8 


Core medicine (R.F.1:2) 


Winte@c) GUciEes" 4.aqbioocdnoosauncoen afatevoreleneteisletereiereretererorenelerelersverctelelercrersieltele oie eis isieereiereseteyeislercrenersye 64.8 62 
England ..cccccccscccses SOOO OOOOOICOIOCICOOOCOOIO OOOO OOOO Omics wees Cece creer ccccscsccvece 14.6 1Die 
CENEGCIE O60 ScmOO G00 HO JO OD OO OOOO OOO mievekelevetencichets (eveholeloraielelsiotsleheleterelcVereleieteislalefele evetonercioretarctovers 55518) 3 
SOoeleel” SG AgnGoscne SOoCmG J U9 O.0 000 O.5 OO CODD 06 DOU OUGM0 DOOD CD DOO OOOO U OUD OO Ibe COnE ad SO OMOOO 2.4 2 
JEER oécooGoneConRAUS OND DODO ONS STOUCODOIDON SOCORDGOUTD OOD OC ODED D0 SOO SH OD. ONIGIS A OMe sad tc tha? 1 
lute UGE Mreletelsceleel suoleteletejerolers © alofelenoler siskelelstelcveicievel helanenetereketeicrelarchereiaeioielers cladarersveleronsienerenchenelel era eiejeretarsiene ee 1 
S WECM Matataletela cts stetelelsrsiels e sletersis sists Mtstelictelotatetcie siere.erertettrale ers enorelenevarcr el oterersietelsl atereteretoverar stare ictoreteneieccrere 1.4 1 
Alister ala anrir ctercres siclenetateletenchetel eter oietichel etehetalictovtel’slefetelletel evelsvelte) clicteiletel eielleretel eve! siete! 01 ele vel oral evelleletetslciel's iolele| ciolele Ar 1 
WEStmGCEMAN Vials ciee's elelsiee ©) Dwiele ce civicwccevceeccecececwesiocccccncccees nodD Oddo doG8 cok SOO UOC Doe) 1 


Core chemistry (R.F.4:3) 


United States ..... 6 Oud CROLROLOOG OGLE, CORSE OOOO B O.B.5 0 COIS O50. OO CCE ROR IR G CRO 0), OO O.OCLLLOLNS 46.9 43 
JEBEIN oc hoses COhG656 06000 SUdN6 GOO5 sSq06600 Gasca oon bo0oHo56C SAOOSOSUOOU.dD DOOO HOOUOUODU ONG 11.0 11 
We Seam GE IN AN Wueales ctedsletarorets) cl speue,chejeusse,e.sx0-010 ele she chaalacetave\ sie slistiejolial elieilelie tala} ellele:s)6l sielle:s lelslisiaiaieleiskols’s|sisisieleleis 8.0 11 
SineN Cll cl evopatateteteretcvetsVercherere olelolilelerelerorstevereys) ole cveuerererelsisrcieierevexedereiereletoie ole/eqs oie enelere feteteionersislelet sists leteisiotele 7.4 
Pit aN e Mieteleiey ofevekersveie evee's\ crete e maloiatatetotelsholcletelalcisistel stelstcioleleletelelcicielsteleleiat=t= aieveleveretereielctelenersteserereteysiererctels 4.8 
SW UZeMEANGM ccc ciele ovsisls Rrolersh eteneroretorevene ae Diet aneeteh ere) teen ctete gusletereneltedote SYokes atelakeholieveterclatorelelslie’eletel cle! steve! sieis 7659) 
SGU! o.omotOROdO OD GOO EGO MeVerstopelel al cfatel cialeleietelele ol sielsicielaioketetelele)ctoleleta! <sielo/el otebeteie ols) eicls oie DOO COO Dol 
healhy a4c66058605006 600004 Soe Cun oOUUD OOOO OdOOuO5bC Soo GOO OGU 6066606 S56 0006 DUO ODOO OO 0 GOOG Way 
TST toUttetenerelevereterseetecehevetetokereisicierelstecertiecsteve. efekexa vie) arereie. elisies0valeie. 6 S66 Hodes 4 suetelaistsvenefeberelelonerecichsterenererele 1.4 
Nemes Vand Seve erevele cretens sielsistevere srelctecsrarel sl els evelettie ersle eters eer eic's\ ee sieretaretencietetercterets cvs aiehevenste claley eevee ekeie hice) 


TABLE 2.3 International Co-authorship Linkages with Canada 
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Poland and only 11 or 1% with the U.S.S.R. (For all research fronts for 1978, U.S. links with East 
Europe were infrequent: 96 with Poland, 61 with Hungary, compared with 1,606 with Canada, 1,377 with 
England, and 998 with West Germany.) 


The profile of Canadian international linkages for R.F.1:1 is somewhat more concentrated: 19 other 
countries, compared with 42 other countries for the U.S. (1978). The striking feature of this profile 
is the concentration of 65% of the linkages with the U.S. The rest of the profile seems somewhat cos- 
mopolitan and balanced: 11% with England, and 4% each with Italy and the Netherlands, and only 2% with 
France, the same as for Denmark and West Germany. Language would thus seem to be a determinant of 
effective collaboration as would geographic location. There are mixed signs for improvement in the 
situation: the range had increased since 1974 from 13 to 19 other countries, but our pre-occupation 
with U.S. linkages actually increased by two points, and a strong set of linkages with West Germany 
(10%) virtually disappeared over the period. 


A possible explanation for this is that while the National Research Council (NRC) slightly expanded 
its scientific exchange program in the mid-1970s the Medical Research Council (MRC) which funds much 
of the Canadian activity represented in R.F.1:1, drastically cut its already modest visiting scien- 
tists program in the same period. Only 13 awards were given in 1972-73 (two foreign) and seven awards 
in 1975=/6. 


Co-authorship data were developed for the other research fronts as well: because of limited space, 
we have chosen only one research front (R.F.1:1) to illustrate how the indicator can be used. 


Distribution of Canadian Publication Activity Among Sectors 


Publication and co-authorship data were also aggregated by means of a standardized dictionary of 
research organizations within Canada. As at the international level, essentially three indicators have 
been generated with respect to the production of papers. These involve (1) changes in the organiza- 
tional production of papers, (2) the analysis of specialization in terms of research profiles which 
distribute that production among research fronts, and (3) co-authorship data among institutions. 


Again, these same distributions will become more qualitative and informative when we insert rele- 
vance values in place of output, as shown in the following chapter, but some illustrative material 
from the output data may be of interest. 


The Canadian share (4.5%) of the world's scientific papers in the 85 research fronts was distri- 
buted among institutional sectors as shown in Table 2.4. This distribution is interesting in several 
respects. The distribution of Canadian activity in terms of publication is concentrated in the univer- 
sities with a strong representation of federal government laboratories as well. A rough comparison 
with the distribution of activity in the United States can be made based on papers monitored for the 
SCI at ISI. The location dictionary used by Computer Horizons, Inc., placed about 68% of publications 
for 1979 in the university sector (including medical schools); results similar to Canadian activi- 
ties. However, industrial institutions in the United States accounted for about 9% of their country's 
activity, while only about 2% of Canadian activity originated in the industrial sector (data not pro- 
vided) .[30] 


Obviously, research in hospitals and clinics is a large and fundamental aspect of medical research 
in this country. It is often pictured as a subsidiary system of clinical teaching arrangements, in- 
ternships, and joint appointments. But there are in fact 173 such research organizations, now publish- 
ing at about the same rate as all federal laboratories (though often in fields with higher publication 
rates, to be sure). Nevertheless, research in clinics amounts to something more than just an appendage 
on a few medical schools, and probably should be treated separately for survey and policy purposes. 
One could explore, for example, the implied research subsidy that must be contained in the transfer 
payment for hospitalization and health care. 


Table 2.4 also includes some important information on possible trends in the dispersion of efforts 
and specialization within these sectors. The dispersion of publishing among university and industrial 
locations, many of which may be ineffectively small, has long been recognized as a problem for 
research policy in Canada. Normalizing these distributions, we can compare the degree of concentra- 
tion, which is maximal for this measure at 0.5. As expected, the degree of dispersion is very great in 
both sectors, but contrary to various policies aimed at specialization and critical size, the disper- 
Sion of research activities would appear to be on the increase. This holds true in the federal sector 
as well, although the units are too aggregated to be particularly meaningful. 


See References at end of text. 
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TABLE 2.4 Distribution of Canadian Activity by Sector 


Sector Papers Units Logi(1) 
numb er pen Mecent number 

1974 
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(1) This is a measure of the concentration of publication frequencies among locations. It varies ona 


scale from 0 to 0.5 such that if all papers tend to come from very few locations, the measure ap- 
proaches its maximum. 


This pattern of increasing dispersion of activity is generally repeated on the world scene for 
those fronts representing very established research areas, whereas the highly dynamic research of 
recent vintage appears to be concentrated in a few laboratories in a few countries. 


Concentration of Publication Activity in Canadian Universities 


Overall publication frequencies for particular institutions are not directly useful, for while 
there is no reason to expect a regional bias in coverage, variations in field coverage and publication 
rates would throw off any comparisons based on such a measure. Space will not permit presentation of 
location output matrices by journal front, but we examined the "research profile" for each location. 
An activity level research profile is a vector, or row from such a matrix, normalized to show the dis- 
tribution of an organization's research among fronts. By way of summarizing the information on such 
profiles, measures of concentration of activity in 1974 and 1978 were computed as well (Table 2.5). 
For example, in 1978 the University of Toronto produced 2,002 papers or 15.7% of the Canadian univer- 
sity total of 12,760 (up from 15.1% in 1974). The University of Toronto's papers were distributed 
among 65 journal fronts in 1978, with a gini concentration of 0.341. An alternate, and perhaps more 
sensitive measure is to take the statistical entropy of the distribution: entropy = 3.586. Despite 
publishing in more fronts than in 1974, this measure indicates that research at the University of 
Toronto became more dispersed in the period. (Zero entries are taken into account here, in order to 
allow the measure to describe a degree of specialization.) 


McGill University was, in fact, one of the most dispersed of all universities in 1974 (gini 0.341 
in 61 fronts); by 1978, McGill was slightly more specialized at 0.361 in 61 fronts. A university 
publishing in 50 fronts or more, with this degree of dispersion, could be placed in the same category 
as Robert Gilpin placed the U.S. and the U.S.S.R. with his term a "research superpower", pursuing a 
world-class level of activity on all fronts. 


The entropy values are perhaps more subtle than the gini, separating out 15 universities with pro- 
file entropies greater than 3.3. All participated widely in the system of research fronts, supported 
by their medical schools with clinical affiliations to sustain their research activities. The 
"TRIUMPH" facility (not shown in Table 2.5) of the University of British Columbia is of course very 
specialized, and was separated as a location along with the university clinics in order to preserve 
accuracy. [TRIUMPH is an atomic research facility jointly administered by three British Columbia 
universities; and it frequently appears as the given address of investigators from those locations. ] 


30 


TABLE 2.5 Concentration of Publication Activity, by University(1) 


Number of Gini Entropy 
journal fronts 


1974 1978 1974 1978 1974 1978 
McGill Umi Versil Vibe eres = ois iene wistersielstorepcneate 61 61 0.341 0.361 3.0 DT4 3.436 
University of British Columbia .......... 60 64 0.343 0.330 Dich 3.648 
University <Of NOPOMUO ticcci so eicateurerss:s a's 58 65 0.344 0.341 3.546 3.586 
University of Alberta 2.2 6ijccocen mee ose os 58 62 0.347 O57 Se 5a 3.602 
University of Western Ontario ........00. oe) 53 0.360 0.348 3.441 3.524 
University of Saskatchewan ....cssscccece 47 48 0.369 0.564 5.914 5.411 
Queen's University at Kingston ......eee. 44 48 aes) O37 5.343 3.346 
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Simon Fraser UMIVErSILY cjs/sclete cate oletoletae 33 33 Oeat> 0.406 Lid, 3.028 
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BrocksUMIVvVersEty cat. < wine elere siejem sini clele aisioietels 16 19 0.444 0.431 2.493 2.689 
University <of. Mani tobias cites wsustere ateveteieuclars < 14 2) 0.440 0.374 bn bo See 
Una Versité de Montreal curse elects civ cters ciare 14 50 0.437 0.364 LoDo Diwttile 
Ecole: Polytechaiquem: sciee ste ee tae iets 14 14 0.447 0.441 2.428 Zeal 
Ste ahrancis XaVilersUnLVErSity Vo. asset. cite 1D 8 0.452 0.465 2.950 Ie S6 
Universitysof Lethbridge <<. s-s1e se ss rie lerts 18 14 0.452 0.445 2.328 2.444 
UnaVeErSiL LEsdeuMONnGEON, ‘swe oo were’s aisteciatate etre 2 9 0.442 0.460 24396 2101 
MEG UpIVERS LEY gcc. </eie stots er sienesinieretn hele cle all 4 0.456 0.444 20220 2.998 


(1) Includes all institutions that published in at least 10 fronts in 1974. 


Interaction Among Authors and Sectors and Distribution of Linkage Activity 


As far as possible, Statistics Canada conventions were followed in allocating organizations among 
sectors so that merging of bibliometric and funding data for further industrial indicators will be 
possible. Thus, among the most prolific industrial research organizations are a number of Crown agen- 
cies operating within the standard industrial classification of industrial sectors. While only six of 
the 172 business organizations publishing in 1978 had a total of 10 or more papers in the 85 fronts, 
only half of them are strictly private-for-profit organizations (Xerox, Ayerst Laboratory and Bell 
Northern Research). The others are Ontario Hydro, IREQ Division of Hydro Québec, and the Pulp and 
Paper Research Institute of Canada (PAPRICAN). Ayerst published in 11 fronts and Xerox in nine; both 
were somewhat dispersed. Bell Northern Research and PAPRICAN were in eight fronts and Alcan in five. 
All others were in fewer fronts than that, and were of course specialized with high gini measures. 


Similarly, the co-authorship data for locations are, for the most part, relevant to research policy 
within particular organizations. However, at the aggregate level, it is of general interest to look at 
the degree of interaction among sectors, as suggested by a measure of co-authored linkages. Several 
policy initiatives in recent years have addressed this area. Table 2.6 gives the overall numbers for 
1978, and percentage allocations for 1974 and 1978. The table again shows the central importance of 
the university sector as a base from which collaborative interaction can be built. By far most of it 
is within the university sector and the health science complex of clinics and other institutions 
surrounding university medical schools. Several federal laboratories have been highly collaborative 
with the university sector, perhaps indicating the importance of some of their research facilities to 
university research, such as those at Atomic Energy of Canada Ltd. (AECL). 


31 


Universities were responsible for even more linkages in 1978 than in 1974, the situation for indus- 
try remained virtually unchanged. This is disappointing because several policy programs have tried to 
establish a closer link between university and industrial sectors for some time. These ties did 
increase absolutely in small numbers, but the resulting pattern remained unchanged. We examined these 
matrices for particular research fronts in applied science, and found the same to be true (data not 
provided). University/business linkages are developing in Applied Mathematics and Computer Science 
(R.F.14:47) for example, but the numbers are still very small. Similar small linkages are beina esta- 
blished for Pharmacology (R.F.3:14, representing food chemistry and pharmacy), and Microbiology and 
Macromolecular Science (R.F.2:15). The numbers may indicate the movement and collaboration of isolated 
individuals, but this type of linkage has obviously not yet been successfully institutionalized.[31] 


See References at end of text. 


TABLE 2.6 Co-authorship Data: Intersectoral Linkages 


Univer- Hospi- Federal Non- Business Provin- Total 
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(1) Includes university clinics. 
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Chapter 3 


AN EVALUATION OF PAPERS 


Methodology 


The evaluation of relevance presented here results from an examination of all references in papers 
monitored by ISI in 1978. All articles, reviews and letters were included, as they do not bias the 
methodology. The largest part of the cost of generating indicators from this data base is initially 
reading the data base itself. Therefore, a minimal selection of cited years was necessary. By examin- 
ing the cited years for each journal, and aggregating this for each class, a reasonable judgment on 
this point led to the selection of 1974 to 1976 for cited years. The measure used to inform this judg- 
ment is called vintage.[32] As an example, in computing the vintage measure for cited years in Core 
Biochemistry (R.F.1:1) we observed that 1978 papers in those journals cited 163 different years 
stretching back to 1543. As expected, the distribution was concentrated on recent years, and, the 
maximally cited year of this very active literature was only two years old. Looking backward from 
1978, some 44,775 references (or 14% of the total) were to papers published after January 1976. The 
"half-life" for most of the literatures was reached at some point within 1975, and rates diminished 
radically behind that point. Therefore, 1974 to 1976 were the years selected, and thus for practical 
reasons 1974 and 1978 became the years for discussing possible trends in the output data in the pre- 
vious chapter. 


Based on their 1978 citation files, the relevance among papers from 1974 to 1976 was computed and 
summed for each paper. Some 1.4 million address records were then searched and standardized (for coun- 
tries, and for locations within Canada). Relevant papers were then allocated to journal classes (but 
the total relevance of each paper to all other papers in the system was retained). For each research 
front all relevant papers with found addresses were rank ordered from greatest to least relevant and 
distributed on a curve against accumulated total relevance for that research front.[33] These curves 
are log-normal Bradford-Zipf type distributions which are very skewed toward a few papers that gene- 
rate most of the relevance. (The usual descriptive statistics based on central tendency would there- 
fore be inappropriate and misleading. ) 


Next the percentage shares of papers by country and organization of origin were computed for the 
whole relevance distribution of each research front, along with the upper decile of relevance, and the 
upper percentile. Even if a country's papers are over- or under-represented in the data base, it is 
possible to examine the change in its share of relevance among the upper decile or percentile of rele- 
vance from its share of relevance for the whole distribution and thus whether its papers in the 
research front may have made a significant contribution. The actual percentage shares for any given 
country are subject to several factors in addition to the quality of research. But changes in those 
shares moving through the distribution to the most relevant papers, represent a key indicator of the 
overall quality of a country's output in that group of journals. In addition, it must be recognized 
that regardless of the overall relevance of a country's contribution, individual investigators may 
contribute very significant work which will be submerged in the overall picture, but nevertheless 
should be examined. (Considerable detail on fine structure is provided at the end of the chapter. ) 


From the analyses which follow, we shall see that those countries contributing the most papers in 
the core research fronts of the system are the same few who manage to maintain and increase their 
shares in the top decile and percentile of the relevance distributions. This pattern repeats itself in 
other research fronts, and leads one to advance the reasonable hypothesis that in order to generate 
very high quality papers, it is necessary though not sufficient to produce a correspondingly high 
overall share of papers.[34] For large research communities, this means maintaining at least a corres- 
ponding share of relevance throughout the distribution. 


For a discussion of international performance, the 26 research fronts with greatest total relevance 
have been selected, in order, without further investigation, to exclude those which might seriously be 
under-represented in the data base and, in particular, to avoid the problem of discussing the perfor- 
mance of a country which may have a strong but unmonitored journal for which its scientists generally 
reserve their best papers. In this case, where we are generally considering clusters of journals from 
the world mainstream, we know that under-represented countries in most cases reserve their best papers 
precisely for these journals. Therefore, if, in the central core of journals, there is a bias in these 
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distributions, it should work in favour of the upper decile performance of peripheral research commu- 
nities. Nevertheless, this means that for international comparisons, we do not consider some important 
fields such as mathematics which were not sufficiently represented in the data base to accumulate the 
necessary share of relevance in the journal system. In the following discussion, the selected research 
fronts are considered as they were grouped into research fields. 


The International Distribution of Highly Relevant Papers for Selected Fronts 


Clinical and Biomedical Research. Throughout the six major medical and biomedical research fronts 
(i.e., Core Biochemistry, Core Medicine, Pharmacology and Clinical Chemistry, Cell Research, Gastroen- 
terology, Surgery, and Immunology and Pediatrics), the United States dominates the picture in terms of 
its control of journals, large output and major presence among the most influential papers. Table 3.1 
shows the international shares of relevance for all countries with at least 1% of total relevance in 
these research fronts, fractionally allocated among co-authored addresses where appropriate. It is 
difficult for U.S. increases in the upper deciles and percentiles to look impressive because of their 
very large initial shares of papers. Nevertheless, while their share of relevant papers in Cell 
Research (R.F.1:6) was 61%, they took 74% of the relevance in the top percentile of papers. This in- 
crease alone is roughly equal to the entire presence of the next five countries. France provided a 
small share of papers which performed extremely well; they were twice as important in the upper per- 
centile compared to the whole relevance distribution. 


Core Biochemistry (R.F.1:1). Although the United States' presence is large in the Core Biochemistry 
research front, the strengths of England, Sweden, Israel and Switzerland are evident from these data. 
Canadian papers overall were slightly less influential than those from other countries and we had less 
than our expected share of the top papers. Specifically, Canada's papers represented 3.6% of all rele- 
vant papers in Core Biochemistry (R.F.1:1) in 1974 and 3.1% in 1978. The Canadian share of papers in 
1974 to 1976 was 3.2% but our share of total relevance for this journal group was only 2.9%. In the 
rank ordered distribution our share of the more relevant papers in the upper decile and percentile 
were 2./% and 2.1% respectively. 


The most influential paper in this research front, and indeed in the entire system of 1974-76 papers, 
was submitted from a research group at the University of Aberdeen, with two collaborating English 
authors. We now know that this paper initiated something of a revolution in many fields of biochemis- 
try and medicine. It is interesting that the first author of the lead paper in R.F.1:1 (J. Hughes), 
also published a related document which turned out to be the most influential in the cluster of main 
medicine journals (R.F.1:2).L35] Because of their great influence and relevance on the whole system of 
papers, the 10 most relevant papers in this research front are identified. 


1. "Identification of two related pentapeptides from the brain with potent opiate agonist activity", 
Nature, 258 (December 18, 1975), p. 577 (J.H. Hughes, T.W. Smith, H.W. Kosterlitz (Research Unit), 
Linda A. Fothergill (Department of Biochemistry, Marischal College, University of Aberdeen), 
B.A. Morgan (Reckitt & Colman Ltd., Pharmaceutical Div., Hull, U.K.), H.R. Morris (Department of 
Biochemistry, Imperial College, London) ). 


2. "Transmembrane control of the receptors on normal and tumor cells", Biochimica et Biophysica Acta, 
457 (April, 1976), pp. 57-108 (Garth L. Nicolson, Salk Institute (San Diego), and Department 
of Biology, University of California at Irvine). 


3. "Chromatin Structure: A Repeating Unit of Histones and DNA", Science, 184 (May 24, 1974), p. 868 
(Roger D. Kornberg, Harvard University, and MRC Laboratory of Molecular Biology, Cambridge, 
England). 


4. "A film detection method for tritium-labelled proteins and nucleic acids in polyacrylamide gels", 
European Journal of Biochemistry, 46, July 1974, p. 83 (William H. Banner, Ronald A. Laskey, MRC 
Laboratory of Molecular Biology, Cambridge, England). 


5. "Cell Surface Proteins and Malignant Cell Transformations", Biochimica et Biophysica Acta, 458 
(April 1976), p. 73 (Richard D. Hynes, Center for Cancer Research, Department of Biology, MIT). 


6. "Thromboxanes: A new group of biologically active compounds derived from prostaglandin endopero- 
xides", Proceedings, National Academy Sciences, U.S.A., 72:8, August 1975, p. 2994 (Mats Hamberg, 
Jan Svensson, Bengt Samuelsson, Department of Chemistry, Karolinska Institute, Stockholm, Sweden). 


7. "Spheroid Chromatin Units (v Bodies)", Science, 183, 1974, p. 330 (A.L. Olins, D.E. Olins, Oak 
Ridge and University of Tennessee). 
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8. "Metabolic Activation of benzo(a)pyrene proceeds by a diol-epoxide", Nature (1974), v. 252, 
p. 236, pp. 326-328 (P. Sims, P.L. Grover, A. Swaislan, K. Pal, A. Hewer, Royal Cancer Hospital, 
Chester Beatty Research Institute, Institute for Cancer Research, Fulham Road, London, England). 


9. "Detection of Specific Sequences among DNA fragments separated by Gel/Electrophoresis", Journal of 
Molecular Biology, 983 (1975), p. 503 (E.M. Southern, MRC Mammalian Genome Unit, Department of 
Zoology, University of Edinburgh). 


10. "Cell surface protein partially restores morphology, adhesiveness, and contact inhibition of 
movement to transformed fibroblasts", Proceedings, National Academy Sciences, U.S.A., April 1976, 
v. 73, pe 1217 (Kenneth M. Yamada, Susan S. Yamada, Ira Pastan, Laboratory of Molecular Biology, 
N. Cancer Institute, NIH, Bethesda, Md.). 


Despite our somewhat disappointing overall performance, a Canadian paper ranked sixteenth in this 
research front. It was contributed by a group in the Division of Neurology of Montréal General Hospi- 
tal Research Institute and McGill University.[36] The complex of medical research institutions in 
Montréal distinguished itself in several less influential journal classes as well. The distinction of 
MRC units in Britain suggests that a close study of their institutional development might be helpful 
in the development of our own research infrastructure. 


Core Medicine (R.F.1:2). Within this journal group the United States again dominates throughout the 
relevance distribution. Compared with the results for Core Biochemistry, Japan and France have rever- 
sed their performance, and along with Sweden increased their shares of top papers. Canadian papers 
maintained their relative shares. We shall examine the performance of individual laboratories in the 
following section, but again, the most influential paper came from the University of Aberdeen 
(J. Hughes, "Isolation of an endogenous compound from Brain with Pharmacological properties similar to 
Morphine", Brain Research, 88:2, p. 295). Clinical affiliations appear to be important, with a small 
group of institutions in Britain and the United States contributing the most important papers. 


Pharmacology and Clinical Chemistry (R.F.1:5). This medical group, consisting of some 29 journals in 
Pharmacology and Clinical Chemistry is in the same research field as Core Biochemistry, but papers in 
these journals did not influence papers in R.F.1:1 strongly enough to allow them to be clustered 
together. Canada does not have a journal in either of these classes, nevertheless, Canadian papers 
maintained their share of relevance and there appeared a high frequency of Canadian papers among the 
most relevant: two papers among the 10 most relevant, and five among the 60 most relevant papers. The 
two most relevant were: 


R.F.1:5:3 "Chemical nature of synoptic transmission in vertebrates" (Physiological Reviews, 54 
(1974), p. 418), by K. Krnjevic, Department of Anaesthesia, McGill University. 


R.F.1:5:10 "Drug-induced prophyrin biosynthesis, XV..." (Biochemical Pharmacology, 25 (1976), 
p. 687), by P.W.F. Fischer, R.O. Morgan, V. Krupa, and G.S. Marks, Department of Pharmaco- 
logy, Queen's University at Kingston. 


It is interesting that also prominent among this list was another paper from the University of 
Aberdeen group: 


R.F.1:5:4 “Purification and Properties of Enkephalin: The possible endogenous ligand for the mor- 
phine receptor" (Life Sciences, 16 (1975), p. 1753), by John Hughes, Terry Smith, Barry 
Morgan and Linda Fothergill, in this configuration, they are identified as a group in 
Research on Addictive Drugs, University of Aberdeen, Scotland. 


Without having done an exhaustive merge of funding records with the bibliometric results, it should be 
pointed out that substantial MRC research grants can be associated with these top Canadian papers in 
R.F.1:1 and R.F.1:5, both before, as well as after, the period under investigation. 


Cell Research (R.F.1:6) and Gastroenterology, Surgery (R.F.1:7). A subsidiary cluster of cell research 
journals (R.F.1:6) again displayed the particular strength of U.S. laboratories, along with those in 
France and Israel. One of the surgery groups (R.F.1:7), with an emphasis on gastroenterology journals, 
is concentrated in fewer countries; Japanese clinics are emphasized. 


Immunology and Pediatrics (R.F.1:8). There was only one Canadian journal among all the journals repre- 
sented by Table 3.1. It is the Canadian Medical Association Journal and it is part of Immunology and 
Pediatrics, a group of 34 journals primarily devoted to immunology, infection and diseases cf chil- 
dren. The upper levels of the relevance distribution of this research front show the strength of 
Canada, Sweden, and more particularly of West Germany. This would seem to reflect the presence of a 
large number of German journals, however, as in Canada's case, this is not so. Germany has only one 
(relatively unimportant) journal in this group; the European Journal of Immunology. 
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Peripheral medical research fronts. Similar taulations for the more weakly connected medical groups 
of journals forming field 2 are given in Appendix V Table 1. The strongest of these, Secondary 
Biochemistry (R.F.2:12), contains 31 journals in biology and psychology though it would be difficult 
to determine a particular subject area. The Canadian Journal of Biochemistry clustered in this group 
as the second strongest journal after the Royal Society Proceedings in Biology. Papers from Canada and 
France did particularly well here. The top paper in the group (Cell Tissue Research (1975) 157:423) 
was produced by a former member of the medical faculty at the University of Sherbrooke.[37] Eight of 
the top 10 papers appeared in the sixteenth journal, Cell and Tissue Research (New York). 


See References at end of text. 
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(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate 
tapes, and those countries with at least 1% of fractionally allocated addresses are shown. 
(2) Had less than 1% papers, but did well with them. 
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The Canadian Journal of Microbiology is in Microbiology and Macromolecular Science (R.F.2:15) along 
with several journals on polymer and macromolecular science. The Japanese have a strong presence here, 
and England and France showed exceptional influence among the top papers. The Canadian participation 
here lacked distinction. 


While Canada's publication level increased by 4.5% over the 1974-78 period within the Clinical Medi- 
cine, Reproductive Biology group (R.F.2:46) our share of world activity declined (from 3.0% to 2.1%). 
Nevertheless, Canada contributed 3.1% of the relevant papers and 4.3% of the top percentile of rele- 
vance for this group. Participation by England, West Germany, Sweden and Denmark are notable. 


Applied Physics. The third field to cluster included the research fronts of Core Physics (R.F.3:4), 
Chemical Physics (R.F.3:9), Solid State Physics (R.F.3:10) and Pharmacology (R.F.3:14). The latter 
class includes journals in toxicology and analytical chemistry. There are two Canadian journals in the 
field: R.F.3:10 includes the Canadian Journal of Physics and R.F.3:14 includes the Canadian Journal of 
Physiology and Pharmacology. Table 3.2 presents the international shares of relevance. 


Canadian papers were strong in both the Core Physics (R.F.3:4) and Solid State Physics (R.F.3:10) 
research fronts. The Core Physics journals also published very strong U.S. and West German papers; and 
while here, as elsewhere, papers from the Soviet Union are not monitored heavily, the importance of 
their contributions is still evident. 


One of the Soviet papers in Core Physics, by A.M. Polyakov of the Landau Institute of Theoretical 
Physics in Moscow, was third in influence. Professor Ajzenberg-Selove from the Department of Physics 
at Pennsylvania published the two most influential papers. Two others came from Cornell, while 
Princeton, MIT and Harvard were represented in the inner 10, along with a large international group 
(France, United States, Italy) based at Berkley and which used the Stanford linear accelerator. 
Canadian addresses were very prominent among the top percentile. Some of these will be noted in the 
following section. The AECL laboratory at Chalk River published the strongest Canadian paper 
(Ropes 24331), one by dale Hardyaandal.c. owners 


As noted in the previous chapter, the level’ of Canadian participation in Solid State Physics 
(R.F.3:10) declined by 4.8% from 1974 to 1978. This group, which includes journals in solid state and 
low temperature physics, quantum electronics and the physics of surfaces, represents an area which is 
important to our electronics industry. Canadian papers showed particular strength in this group, and 
the declining activity might turn out to be costly. The United States, Japan, and England were strong 
performers in this area. The exceptional influence of papers from Czechoslovakia in chemical physics 
and applied chemistry should also be noted. 


Some "noise" was introduced into the analysis of physics papers. A surprising degree of cited work in 
particle physics (part of Core Physics) is in the form of abstracts. This could be due to a number of 
factors including the delay time for formal articles, the depth of acceptance and understanding of 
given procedures within well formed invisible colleges as well as the rapid development of work in 
this field. In any case more than one influential abstract might appear on the same page, inflating 
the relevance evaluation of both to some extent. This source of noise is random and does not predomi- 
nate over normal papers, hence the overall shares should not be altered (although co-authorship data 
for Core Physics would be misleading). 


Core Chemistry (R.F.4:3). It was noted before that Core Chemistry, the third strongest cluster of 
journals in the system, formed a research field by itself, exerting wide influence throughout the 
system. This group (see Table 3.3), represents an area of traditional strength for West Germany, going 
back to the institutionalization of university-industry linkages in the last century. It had 9% of the 
relevant papers for 1974-76, 15% of all relevance and 25% of relevance in the top percentile. Britain, 
with a much smaller share, also displayed particular strength among the more relevant papers. Canada 
has a strong journal in this class, and contributed a substantial share of papers, but they did not, 
as a group, influence the rest of the system significantly. Canadian output declined by 3% in the 
period 1974 to 1978, while West German output increased by 43%. Five of the top 10 papers came from 
West Germany, and there were contributions from the U.S., the U.S.S.R. and Japan. 


Without formulating the argument in detail, the reader is reminded that many of the numbers presented 
in this chapter seem to support the hypothesis mentioned earlier (i.e., that it appears to be neces- 
sary to field significant numbers of researchers in order to cultivate excellence among them). "Culti- 
vation", however, is a term that conceals a large number of institutional and policy variables. 
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TABLE 3.2 International Shares of Relevance in Applied Physics, 1978 Reference Year 


Share of Share of Share of Share of top 


Research front/country relevant relevance top decile percentile 
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Chemical physics (R.F.3:9) 
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Solid state physics (R.F.3:10) 
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(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate tapes, and those coun- 
tries with at least 1% of fractionally allocated addresses are shown. 
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TABLE 3.3 International Shares of Relevance in Core Chemistry (R.F.4:3), 1978 Reference Year 


Share of Share of Share of Share of top 
Country relevant relevance top decile percentile 
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(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate 


tapes, and those countries with at least 1% of fractionally allocated addresses are shown. 


Chemical Analysis and Plant Biology. The four research fronts comprising field 5 (see Table 3.4) 
involve areas of Chemical Analysis and Plant Biology. This is a traditional area of strength in Cana- 
dian agricultural research. Though we have no journal in the Physical Chemistry group (R.F.5:20) our 
papers in chemical analysis and spectroscopy were particularly strong, as were papers from both East 
and West Germany. In comparing the results of this group with Physical Chemistry: U.S.S.R. (R.F.5:25) 
the effects of the relative isolation of journals and research in the Soviet Union are apparent (at 
least in this area of chemical analysis). Even though the Soviet journals are frequently translated 
they almost define their own research front. Language barriers do not appear to have resulted in simi- 
lar separations among western countries. Occasionally there are areas, like Physical Chemistry, which 
include strong participation from Eastern Europe. The Soviet Union is not strongly enough represented 
in the data base to examine this aspect thoroughly. 


To conclude this section on international performances in research, the following comments for spe- 
cific research fronts are notéd and the reader is directed to Appendix V for the supporting data. 


Main Mathematics (R.F.6:17) (data not provided) under-represents the Soviet Union, much of Europe 
and many of the newer journals and therefore, it did not generate sufficient relevance to be among the 
groups considered internationally. However, we note that Canada's level of activity and world share 
declined significantly despite the strength of its papers; we had 5.1% of total. relevance and 11% of 
the top percentile of relevance, a paper from the University of British Columbia placing second among 
the most relevant. 


The cluster of Applied Mathematics, Secondary Physical Applications (R.F.6:30) includes several 
international mathematics journals. If coverage had been more complete, one assumes that this cluster 
would have become differentiated in the analysis. The United States and Italy are exceptionally strong 
in this group (see Appendix V Table 2). However, it does contain several specialties, and several of 
the most influential papers are in plasma physics. The citation characteristics of this area may be 
sufficiently different from the papers in applied mathematics to have overshadowed their importance. 


The journal group, Chemical Engineering (R.F.7:34) (Appendix V Table 3) contains a strong contr ibu- 
tion from India. Canadian papers were particularly influential in Photo-chemistry, Applied Physics 
(R.F.7:41) covering photo-chemistry, mass spectrometry and computer physics. 


Geology and Astrophysics (R.F.8:13) contains the main journals in geophysics, which have been 
joined in recent years by journals in astrophysics. The importance of space sciences for geology no 
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TABLE 3.4 International Shares of Relevance in Chemical Analysis and Plant Biology, 1978 Reference Year 


Share of Share of Share of Share of top 
Research front/country relevant relevance top decile percentile 
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(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate tapes, and those coun- 


tries with at least 1% of fractionally allocated addresses are shown. 
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doubt upsets the traditional field in which Canada once predominated. Now one finds particular influ- 
ence from the United States, England and Australia (Appendix V Table 4). 


Western Europe has a greater presence in the area of materials research and applied physics 
(Physics of Materials (R.F.9:23), see Appendix V Table 5), with particular influence from England, 
France, Sweden and the Netherlands. The Canadian share of papers here was large, but their influence 
was negligible. Subsequent to the time period under study, our modest university efforts were supple- 
mented by the development of federally supported laboratories in this area. 


Applied Mathematics and Computer Science (R.F.14:47) contains some 37 journals in computer science 
and related mathematics, but again, it did not generate sufficient relevance to be considered a solid 
base for international comparisons. It is noted for Canada that despite our world share drop in 
published papers we showed particular strength in the papers contributed: we took 7.6% of total rele- 
vance, held on to 6% through the top percentile, and contributed one of the inner 10 papers (data not 
provided). 


Finally, Applied Statistics, Population Biology (R.F.10:50) contains a large mixture of journals in 
food science, biometrics and other statistical applications. Papers from the United States performed 
strongly in the distribution, along with those from England, France, Australia and Denmark. Canada had 
a substantial share and maintained this throughout the relevance distribution (see Appendix V Table 
Gis 


This completes an overview of the research fronts considered suitable for an evaluation of papers 
on an international basis. We turn now to a closer examination of Canadian activities, to which some 
reference has already been made. 


In Search of Excellence in Canadian Science 


The Canadian share of relevant papers (for which addresses were found) for 1974-76 amounted to 
28,410 papers or 4.5%. In an overall distribution of relevance, 2,851 Canadian papers were in the top 
decile and 252 in the top percentile. For this part of the analysis of Canadian activity, the rele- 
vance factor for each paper was fractionally allocated among all given addresses. For all research 
fronts, this fractional relevance was then aggregated for each standardized Canadian address over the 
whole distribution, and then over the top decile and the top percentile of the curve. These values 
thus sum up to the values given for Canada as a whole in the previous section. 


Since the frequencies of most important papers by Canadian organizations in any given research 
front may be small it is more useful to give a ranking of institutions in the top percentile of rele- 
vance and the actual frequencies of address records rather than percentage shares as in the interna- 
tional presentation. All institutions appear if they have at least part of a paper in the top percen- 
tile of world relevance for the class. 


We have noted in previous discussions that while Canadian shares of relevant papers were substan- 
tial in most research fronts along with shares of total relevance, in only seven of the 32 interna- 
tional research fronts did we substantially increase our expected share of the upper decile and per- 
centile of relevance. On the surface this would seem to narrow considerably the search for overall 
excellence in world terms. In any case, as a first approximation in the search strategy we may search 
for data on our stronger institutions by looking first at these few research fronts where Canadian 
activity as a whole was taken particularly seriously. Complete data on the performance of Canadian 
Laboratories in the top decile of relevance can be found in Appendix V Table 7. 


Core Biochemistry (R.F.1:1). Within the area of Core Biochemistry, some 88 separate Canadian organiza- 
tions contributed relevant papers; the University of Toronto appeared most frequently (16.6%) followed 
by McGill University (8.5%), University of Alberta (8.4%) and the Ottawa laboratories of the NRC 
(4.8%). However, McGill had a very strong paper and this was sufficient to put it first in a ranking 
of institutions for the very top percentile of relevance in this group. Actually AECL appeared most 
frequently in the top percentile, but the fractionally allocated relevance of its five papers did not 
equal that of the single McGill paper which ranked sixteenth in world relevance. The AECL paper ranked 
number 140 in the world research front. Their respective organizations ranked two and 13 in the top 
decile, and one and four in the top percentile. No effort has been made to normalize these results 
with the numbers of investigators, or the numbers of overall contributions in the front, but none of 
the laboratories in the top percentile is small or peripheral in the Canadian community. Table 3.5 
shows the ranking of the top seven institutions in R.F.1:1. Details on the 32 institutions with papers 
in the top decile can be found in Appendix V Table 7. 


While one would expect these areas of medicine and biochemistry to be dominated by those universities 
with medical schools and associated teaching clinics, it is notable that two federal laboratories are 
among the select seven institutions. The Ontario Cancer Institute also appears frequently as an 
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address of exceptional papers in many of the fronts of medical journals. While we do not often find 
isolated and small facilities, it can be seen from these and results to follow, that the highest 
quality of output is not at all a simple function of size, level of output or level of funding. 


TABLE 3.5 Canadian Institutions in the Top Percentile of Relevance in Core Biochemistry (R.F.1:1), 
1978 Reference Year 


Overall fre- Frequency Frequency 
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decile percentile 
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(1) Frequency refers to the number of appearances of an organization's address. All addresses on each 
paper are taken into account and the relevance measure is fractionally allocated. 


Pharmacology and clinical chemistry (R.F.1:5). In R.F.1:5, one can find many papers with titles simi- 
lar to the most influential in Core Biochemistry (R.F.1:1), but they do not attract the same wide 
attention here. For example, the Hughes group at Aberdeen had produced the top paper in R.F.1:1 which, 
as noted, initiated a large response throughout the system. Hughes alone also wrote a similar and very 
strong paper in Core Medicine (R.F.1:2). In R.F.1:5, the same group produced another paper on enkepha- 
lin and the releasing mechanism for morphinelike substances from the brain. This time, publishing in 
Life Sciences, a British clinical journal, they perhaps reached a distinctive group within the large 
community of biomedical research, and the paper turned out to be the fourth most influential in the 
research front. It was nosed out this time by a McGill paper "Chemical Nature of Synoptic Transmission 
in Vertebrates 2", by K. Krnjevic (Physical Review, 1974/54/418). A group from the Department of 
Pharmacology at Queen's University, Kingston, produced a paper ranked No. 10 (Biochemical Pharm., 
1976/25/687). This again shows that even two years is sufficient time for the relevance measure to 
pick up and indicate the impact of a paper in such a dynamic area (it also shows that the measure is 
not sensitive to the size-of-citation-file problem that plagues other impact measures). The fact that 
Queen's University has four appearances in the upper percentile is due to a co-authorship of four on 
their strongest paper.|38] Business also puts in an appearance in this front with a paper from Ayerst 
(pharmaceutical) Laboratory, Montréal. The distance of Canadian researchers from industrial pharma- 
ceutical laboratories would seem to be costly here as well as in more fundamental areas of bioche- 
mistry. 


Four of the top 30 papers in journals of pharmacology and clinical chemistry were by Canadian univer- 
sity staff. Some 94 Canadian organizations contributed relevant papers, though the distribution of 
relevance was quite concentrated among them (logi = 0.409, entropy = 3.095). McGill again produced the 
strongest showing in both the top decile and percentile, this time from a base of a higher level of 
activity, more in line with the University of Toronto. It is shown here again that performance is not 
a simple reflection of size. Some 31 organizations had a level of activity equal to or greater than 
the Ontario Cancer Institute, and the University of Windsor. However, while both institutions were 
among the smallest contributers in numbers, both were among the strongest in terms of relevance. Table 
3.6 shows the ranking of the top seven institutions in R.F.1:5. 


See References at end of text. 
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TABLE 3.6 Canadian Institutions in the Top Percentile of Relevance in Pharmacology and Clinical Chem- 
istry (R.F.1:5), 1978 Reference Year 


Overall fre- Frequency Frequency 
Canadian rank/institution quency (1) in top in top 
decile percentile 
1) Intec WL WimwerSey pootooonooDoCGnDOOS 60 | 158 y 1 
2 Queen's University at Kingston ......... 43 9 4 
3 University of Saskatchewan ........ coe Z> 7 1 
4 URNIVEESIEY Of TOROMEO war eres ais'o oles tele once Oo 23 Z 
> Ontartoy Cance ne nS Gucuitce sms OL OMUGmerstelarcs 7 1 1 
G UNIVERSITY, Off WANGSOE ais. oe wele-sie eis ione wietele 3 1 1 


(1) Frequency refers to the number of appearances of an organization's address. All addresses on each 
paper are taken into account and the relevance measure is fractionally allocated where appropri- 
ate. 


Secondary Biochemistry (R.F.2:12). The journals of cell biology, protein chemistry and physiological 
psychology represent an area in which Canada had a large world share and increased that share appre- 
ciably in the upper relevance percentile. 


There were 70 organizations involved in this front. Relevance was somewhat dispersed (gini = 0.358 and 
entropy = 3.255). The Université de Sherbrooke's participation level was not large (there were 16 
organizations with more papers) but the top world paper in this journal front was produced at 
Sherbrooke. Table 3.7 shows the ranking of the nine institutions in the top percentile of R.f.2:12. 


TABLE 3.7 Canadian Institutions in the Top Percentile of Relevance in Secondary Biochemistry 
(R.F.2:12), 1978 Reference Year 


Overall fre- Frequency Frequency 
Canadian rank/institution "quency (1) in top in top 
deci ke percentile 


1 Universitéede Sherbrooke msjews «ic aks epee see's 16 4 1 
Z5UnIVeRSsd ty Ofe TOL OnUOp inj srt ieicl siete ss LES 12 Z 
5, Carleton clini vers Uy: Gt sitisiiates es sysctl tetslo mee 4 1 
4 University esofeWesterm Onbard0ge e's sete ete D2 4 fr 
oy Sieky Childrens hospital, wl onontomee. s,s Zz 3 1 
GeUNEVersut Gy ltavalats scheviar. cmraele del tetetelarsters 24 10 1 
7Ontario: Cancer Institute; laoronton. s..< 2 1 1 
8 Laval Centre Hospital ...... svctstte otele ae stats 1 1 1 
PROUMONGE LASEL| UM LVEESAL Yor eeree cielo o stelare tetere ite 1 1 


paper are taken into account and the relevance measure is fractionally allocated where appropri- 
ate. 


One could speculate that the high performance levels of such smaller research centres as the Ontario 
Cancer Institute and the Laval Centre Hospital is due to their involvement with much larger and com- 
plementary research complexes. Thus, high levels of activity might be maintained, for example, in the 
context of teaching and health care functions at the University of Toronto and the Sick Children's 
Hospital, while specialized centres in the same complex may become a focus for excellence. In any 
case, the influence of Laval papers was also strong in relation to its overall contribution. 


Clinical Medicine, Reproductive Biology (R.F.2:46). Canada had 3.1% of the papers, and 4.3% of top 
percentile relevance in the much more loosely connected research front of R.F.2:46, which contains 
journals in clinical endocrinology, gynecology and cell genetics (Table 3.8). Some 44 Canadian 
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organizations contributed relevant papers, but only two of them were in the top percentile: McGill 
University and the University of Manitoba. Though there are 49 loosely connected journals in the 
group, representing several research areas, publication and citation rates would appear to be low. 
Toronto only had 30 appearances in total, followed by Western with 16 and Manitoba and Sherbrooke with 
14 each. The performance of other organizations not in the top percentile can be found in Appendix V 
Table 7. McGill managed to rank highest in the top percentile on the base of just 11 appearances, and 
this seems to be characteristic of its excellent performance in several medical fields. 


TABLE 3.8 Canadian Institutions in the Top Percentile of Relevance in Clinical Medicine, Reproduct ive 
Biology (R.F.2:46), 1978 Reference Year 


Overall fre- Frequency Frequency 
Canadian rank/institution quency (1) in top in top 
decile percentile 
feMci Wien TYCL SMO] sarees stats cclets e tisiets ss eisteun) 1.1 3 1 
PUM UV CL SE LY sOn \ManUt ODidtess, s/o ere aisle ve eie ce 14. 1 1 


(1) Frequency refers to the number of appearances of an organization's address. All addresses on each 


paper are taken into account and the relevance measure is fractionally allocated where appropri- 
ate. 


Leaving medicine and passing on to the research fronts in field 3 concerned with applied physics, one 
can again observe the overall excellence of Canadian contributions, particularly those from government 
laboratories. 


Applied Physics. Tables 3.9 and 3.10 present data for Core Physics (R.F.3:4) and the key group of 
journals in physical chemistry, quantum electronics and mathematical physics (R.F.3:10). 


We repeat a note of caution in interpreting the top percentile of Core Physics: the citation of many 
papers in the form of physics abstracts has introduced some noise (which we believe to be random) 
where two exceptional abstracts might have been published on the same page. This possible source of 
error might be misleading for very small groups of articles, such as in the top percentile. Neverthe- 
less, the ordering of locations is probably accurate. Assuming this to be true, physics appears to be 
in good shape at all of these locations, particularly at McMaster, AECL and Laval. 


TABLE 3.9 Canadian Institutions in the Top Percentile of Relevance in Core Physics (R.F.3:4), 
1978 Reference Year 


Overall fre- Frequency Frequency 
Canadian rank/institution quency (1) in top in top 
decile percentile 


1 McMaster University ..ccccccccsccccccscccccscevesvece iy ? 6 
UMA VEDS LL GMIEMMONLE CALM sie tereterc'e wis seisic erec's cleveds cee ees 42 2 Z 
3 Atomic Energy of Canada Ltd., Chalk River, Ont. ..... 124 7) 1 
GMINiversitelVavall sess Se eee cls cwiee ewe ccce cles eclce 23 4 Z 
DEMGGULPURTVETSIUY, Foote sere sect ec ceeede sieeve ctw ee | AD 5 1 
GeUniverssvy 2o fe ManiUob ase. We. Le vee wclnle a slelee c's siete 59 4 1 
PAMINELSTUVSOTETOLOMUOW. weiss cee ec clstere ie late wee clo sieesemint zo 12 2 


(1) Frequency refers to the number of appearances of an organization's address. All addresses on each 
paper are taken into account and the relevance measure is fractionally allocated where appropri- 
ate. 


In R.F.3:10, Canada seems to have demonstrated considerable depth and influence throughout the rele- 
vance distribution, involving laboratories from all sectors. We had 16 of the top 100 papers (ranked 
NOSee? 1B le oe woreda AS ee ObeE 2, DO,859,°6>, (7g 2,°078, 785 )eRelevantresearchin this*area 
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of applied physics was distributed among 6/7 Canadian organizations and this again is comparatively 
dispersed (gini = 0.377, entropy = 2.982). The University of Toronto, NRC, University of Windsor, and 
AECL recorded strength, but the fact is most notable that 29 Canadian organizations were in the top 
decile from all sectors, including six business firms (see Appendix V Table 7). 


TABLE 3.10 Canadian Institutions in the Top Percentile of Relevance in Solid State Physics (R.F.3:10), 
1978 Reference Year 


Overall fre- Frequency Frequency 
Canadian rank/institution quency (1) in top in top 

decile percentile 
PUNT VErSiEVveO fs NOLO LG mise crorete espera avageiststetelctclersis<istsiaicei smn Cu ce, 4 
2 Natftonal Researelh ‘Council, Ottawa. scene ss access tO 14 4 
S University Ot WINdSOL Serie alee s sles. visiecisis' ee 6 ele seees 18 3 1 
4 Atomic Energy of Canada Ltd., Chalk River, Ont. .... Dil y 2 
5 UNIVErSity on SaSkateMnewai am oc ss) sisleis 4 eis cisreretsiatelelorete a » Z 
6 Queen's University at Kingston ...... SSewwee stoiel ouerels Mags) 5 1 
7 Trent University jos teens, Dene te tae eS es 8 2 Z. 2 
B University of WesterneUme arta ae. ete etoile esse BAGS. con 38 8 5 
? McMaster UNL VErSItY see vise eas alate evi eye oaieth ofclieges se acisr stator Ut 14 Z 
{OieUnAiversité- des Montreal gi acjete ete cis eretete sts wietenetelelnistats tater yer 48 9 2 
1 University 10 fa Mani LODa om curs ssiplete stetele siete cfeterois <istalete of 2D D 1 
LZ University of OCGaAWa aes. ctetscels ofa) «sfc olels ctetel sielcbevers or eieteienile 13 1 1 


1) Frequency refers to the number of appearances of an organization's address. All addresses on each 
paper are taken into account and the relevance measure is fractionally allocated where appropri- 
ate. 


As shown in Table 3.10, the dozen organizations in the top percentile contain no industrial firms, and 
are led by strong physics divisions in two federal laboratories and two universities, Toronto and 
Windsor. In addition, one observes a particularly strong performance of papers from the University of 
Saskatchewan on the basis of only seven appearances. 


In Pharmacology (R.F.3:14) we again see the strength of government laboratories in environmental and 
health-related regulatory areas. The top paper in the world in this front came from the Canadian 
Centre for Inland Waters at Burlington.[39] (See Appendix V Table 7.) 


Tables 3.11 and 3.12 present similar information for the other two research fronts in which Cana- 
dians had substantially increased their share of top relevance: Physical Chemistry (R.F.5:20), and 
Photo-chemistry, Applied Physics (R.F.7:41). Both of these research fronts include journals in applied 
physics and chemistry involving instruments for molecular spectroscopy and mass spectrometry. R.F.5:20 
is a very cosmopolitan front, but perhaps because of the specialized instruments involved, only 37 
organizations contributed relevant papers (gini = 0.321, entropy = 2.871). The University of British 
Columbia contributed most of them (40), but only five were in the top decile and none in the top per- 
centile. The Universities of Toronto and Western Ontario were next, but the numbers are very small. We 
have already noted that scientists at these universities publish fairly extensively in main physics or 
main chemistry, and not so frequently in this more applied or chemical analysis group of journals in 
photo-chemical applications. In any case, papers from Waterloo and Manitoba performed well for their 
small numbers, with strength indicated for the chemistry departments at NRC and Guelph. 


See References at end of text. 
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TABLE 3.11 Canadian Institutions in the Top Percentile of Relevance in Physical Chemistry (R.F.5:20), 
1978 Reference Year 


Overall fre- Frequency Frequency 
Canadian rank/institution quency (1) in top in top 
decile percentile 
Ie@Universityseof Water lno?)..acu% 4 ce st) ve 2 
BRUEVEE SIL YLO Geialic Mh. . srainate ss ce s.c cess 941 | 8 2 
3 National Research Council, Ottawa ...... 13 7 1 
aulniversityvof Manitoba wisn. suis. abe oheile 7 Z 2 
pniversity oh Torontomsye. 6... ale. i « note cz 7 


(1) Frequency refers to the number of appearances of an organization's address. All addresses on each 


paper are taken into account and the relevance measure is fractionally allocated where appropri- 
ate. 


TABLE 3.12 Canadian Institutions in the Top Percentile of Relevance in Photo-chemistry, Applied 
Physics (R.F.7:41), 1978 Reference Year 


Overall fre- Frequency Frequency 
Canadian rank/institution quency (1) in top in top 
decile percentile 
MeUniverstbymof OUGawa Ciaaic. 1). Sere o cble onic 7 3 3 
PRUMIVES SUL YS OT LIOFOMLO) .0'ess > os else olotsiohe fst 20 7 2 


(1) Frequency refers to the number of appearances of an organization's address. All addresses on each 


paper are taken into account and the relevance measure is fractionally allocated where appropri- 
ate. 


This search for excellence in Canadian papers would seem to be most promising for areas where the 
class of journals is fairly homogeneous, and within or close to the mainstream of world research in 
their field. It is no particular consolation to publish the strongest papers in a subsidiary class of 
journals that the world treats as peripheral. However, in this section, we have thus far only searched 
in research fronts where Canada as a whole has done well, and this undoubtedly means we have cast the 
net too narrowly. It remains always possible that where Canadian papers perform poorly taken all 
together, some few might still prove exceptional. Moreover, since we are now only looking at Canadian 
locations, we can be interested in all research fronts where Canadians publish substantially, even if 
some foreign countries or some research specialties are not well represented in those fronts. 


Therefore, Appendix V Table 7 provides similar data, for all research fronts in which Canada at 
least participated in the top decile of relevance. The research fronts are presented by fields. The 
data are organized essentially in the same form as Tables 3.5 to 3.12, except that rank and frequency 
are provided for both the top decile and the top percentile of relevance. Again, with respect to the 
location dictionary, hospitals and clinics were separated from universities for consistency, even when 
their affiliation with the medical school is strong. Every address included on each paper was included 
here; the frequency counts are for appearances of that address in a particular part of the relevance 
distribution. 


It is easy for the Canadian reader to scan the organizations on Appendix V Table 7, comparing 
expectations with results. Surprises come up in each research front. For example, in R.F.1:6 covering 
cell biology and cancer research, Canada had papers ranked 24, 60, 81 and 85. The University of 
Toronto with 17 appearances, ranked first in the top decile, produced only one of these papers (No. 
85), and ranked fifth in the top percentile. In fact, researchers at Université Laval produced two 
(No. 24 and No. 81)3 University of British Columbia had another. The small but excellent contribution 
from the Ontario Cancer Institute is notable again in this research front. 


In Core Medicine (R.F.1:2), Canada had three of the top 100 world papers (the best one, No. 3, 
coming from the University of Calgary). This single paper was just enough to place Calgary among the 
10 Canadian organizations in the top percentile of world relevance (co-authorship forces some of the 
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relevance of exceptional papers to be assigned elsewhere, hence the Université de Montréal, also with 
a single paper, placed fourth in the top percentile). 


From the point of view of searching for excellence in research organizations as opposed to single 
investigators, one could look for a reasonable level of appearances in the top decile, and then com- 
pare the share of top decile papers that make it into the top percentile. Thus, for the previous 
cluster of Core Biochemistry (R.F.1:1), all five of AECL's top decile papers were also in the top per- 
centile, while only one in the 18 from McGill. In Core Medicine, Toronto's Hospital for Sick Children 
had three in eight, and the Royal Victoria Hospital in Montréal had three in 10. But, should one con- 
sider it a mark for or against the Royal Victoria that they had two extra papers in the top decile? 
One's answer to such a dilemma will at the same time betray one's assumptions with respect to the 
Ortega Hypothesis (i.e., Reference 34). 


In Core Chemistry (R.F.4:3), despite a disappointing national performance, we had four papers in 
the top 100 (Nos. 12, 14, 27, 59), and nine universities had 10 or more appearances in the top de- 
cile. The surprise here, in view of their funding and size, is the strength of the Universities of 
Alberta and Western Ontario, in a-chemistry class dominated by physical chemistry. On an individual 
basis we have not found that funding levels in this area correspond with performance. 


Though we could not discuss mathematics in an international context Main Mathematics (R.F.6:17) 
does contain 12 important international journals in this field. Canada produced a very significant 
paper here, ranked second in the world (Transactions, American Mathematical Society, 205 (1975), 
p. 247); "Generalized Gradients and Applications", by Frank Clarke at the University of British 
Columbia. 


At the same time, the data for Geology and Astrophysics (R.F.8:13) seem to confirm that our periph- 
eral relationship in astrophysics and space sciences has pushed Canada out of its previously strong 
position in geophysics. 


In Applied Mathematics and Computer Science (R.F.14:47) as in Main Mathematics, we had a single 
excellent paper, also from the west, this time the.University of Victoria: I.) Bbacrodale, ai. Dake 
Roberts ("Solution of an Overdetermined System of Equations in the 1, Norm (F4)", Communications 
ACM, 1974, 17:6). In computer science, the usual assumption has been that MIT, Stanford, Toronto, Case 
and Waterloo dominate the world, roughly in that order. But aside from the excellent paper from 
Victoria, it may be noted that Anthony N. Michel from the state University at Ames, Iowa was involved 
in the authorship of three of the top six papers in this group. The top paper, "Input-Output Stability 
of Composite Feedback Systems 2" (M. Araki, IEEE Transactions on Automatic Central, April 1976), was 
produced at Kyoto University in Japan. 


Chapter 4 


SUMMARY 


This report describes the three main results of a study in bibliometrics involving an evaluation of 
the research papers and journals monitored by the ISI: 


. the development of a new measure of the quality of research in science and technology; 


. the development of a procedure for using that measure, called relevance, to perform a structural 
analysis and evaluation of research literature; and 


- the implementation of the procedure using data from the information systems of the ISI with the 
results of a pilot-study of the implementation of these procedures. 


The analysis of the system of journals resulted in a series of controlled classes of journals 
(i.e., research fronts) for which a number of bibliometric indicators could be tallied. Publication 
trends, for example, in terms of national shares of output in key areas, were presented. Co-authorship 
trends between countries and sectors were also investigated. Measures of the dispersion of publishing 
activities among countries and laboratories are to be contrasted with the high concentration of excel- 
lent papers in relatively few laboratories and institutions. 


In some few areas such as Solid State Physics (R.F.3:10) one can find an impressive range of Cana- 
dian laboratories from all sectors contributing excellent papers. In virtually all areas, isolated 
individuals can be identified within the higher percentile of relevance in their respective communi- 
ties. But it could hardly escape one's notice that the overall performance of Canadian laboratories is 
disappointing in many critical areas. These results are symptomatic, and would have to be related to 
many factors in order to discover their full implications. 


Clearly, the context of the historical evolution of our research institutions, still very much in 
process, is most relevant. Some of this context can be identified: the mechanisms of the expansion of 
activities within universities; the orientation and management of government facilities; the emergent 
and isolated character of industrial research; the difficulties of the linkage among institutions; the 
inertia of evaluation panels and networks in the system of direct funding; instrumentation; informa- 
tion support; and journals. The quality of output from research activities is clearly not a simple 
function of size or funds. These are some of the intervening variables in the search for excellence. 
Without exception, they involve the difficult but historically creative processes of institution 
building, and we know that these processes were delayed in Canada by our neo-colonial attitude to the 
much more highly developed institutions in Britain and the United States. Critical studies in the in- 
stitutional development of our research infrastructure are by now long overdue. 
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Scientific Output" in Y. Elkana et al. (eds.), Toward a Metric of Science, Wiley (New York, 
VOTE )ewaps 197. 


A good summary of the theory is William Goffman, "A General Theory of Communication", Tefko 
Saracevic (ed.), Introduction to Information Science, p. 726. 


See Appendix I. The analysis of relevance in the sense used here was initially developed by the 
author for a project in the Complex Systems Institute at Case Western Reserve University (the 
Information Utility Project, Jan. 1973). Preliminary applications used closure in a distance 
matrix to partition the source, reported in MacAulay, "Analyzing Formal Sources in a Complex 
Information Utility", American Society for Information Science, Proceedings (Atlanta, 1974); an 
application to a university faculty is in MacAulay, The Communications Analysis of a Knowledge 
Community, Ph.D. Dissertation, C.W.R.U. (Cleveland, 1976). The large-scale application reported 
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here was developed as a pilot project in Science Indicators for the Science Council of Canada, 
and reported initially in MacAulay, "A Family of Research Indicators Derived from a Bibliometric 
Analysis", Proceedings, Science and Technology Indicators Conference, OECD (Paris, 1980). 


An application in ibid. (MacAulay, 1974), involved only some 35 biomedical engineering staff; 
using a similar strategy, Computer Horizons clustered some 103 physics journals (Evaluative 
Bibliometrics, op. cit., p. 206); an efficient algorithm such as the one used in this study runs 
into theoretical limits on present computers at aout 20,000, which is excellent for journals, 
but still far too small for the direct analysis of papers. 


Eugene Garfield computed his impact factor for Canadian journals in 1977 Current Contents, 33 
(Aug. 13, 1979). For example, the Canadian Journal of Chemistry ranked number 70 on his list in 
terms of total citations, but was demoted to number 494 in terms of impact (behind Canadian 
Journal of Biochemistry, only 311th in citations, but 412th in impact). The impact factor is 
seriously biassed because it misses such other factors as the type of papers in the journal 
(review papers, or letters), their transdisciplinary content (e.g., influencing fields with high 
citation characteristics), the fact that all citing journals are not the same quality, and the 
excessively incestuous citation behaviour within mainstream countries like the U.S. In general, 
such a measure takes seriously the idea of measuring a flow of information rather than looking at 
resultant relations of influence. Citations alone can mean many things in addition to a flow of 
information. Nevertheless, in its policy paper on Canadian journals, a committee of the Royal 
Society used these data directly as a measure of quality (The Press of Knowledge, Royal Society 
(Ottawa, 1978)). More recently, the Bovey Commission on the universities of Ontario committed 
itself to a similarly indefensible measure. 


The data base separates addresses in Scotland and Wales from England, as for Soviet republics, 
and since Scottish research as distributed among journal clusters, represents a very different 
profile from that in England, it is useful to retain the separation. 


The industrial share was down from 11% in 1973, and for a very general eight-field allocation of 
journals, U.S. industry produced 36% of papers in engineering and technology (National Science 
Board, Science Indicators - 1980 (Washington, 1981), p. 251). The U.S. dictionary does not cover 
small laboratories producing fewer than 10 papers/year; therefore these data understate the 
industrial comparison. 


The institutionalization of inter-sectoral research activities, with special reference to pilot 
plant facilities involving university, industry and government, was examined in Machine in the 
Garden, Discussion paper, J. MacAulay and P. Dufour, Science Council of Canada, 1984. 


The “currency” or “vintage” in the use of the past years of a literature can be used as an accu- 
rate overall indicator of its dynamics. Dr. Lam derived a measure for the contribution of year 1 
to any particular vintage: V, = (% of all 1978 cites attributed to year i) X (1979 - year i); 


n 
and thus the vintage character of the kth journal front is V, = Se ave 
=| 


for n cited years of literature. The cited year column is the only aspect of the cited body of 
science which can now be read in its entirety. Thus, while the vintage measure may have some 
peripheral interest to historians, it is actually the most critical measure available to confirm 
the boundaries of research areas. 


There is a slight time lag for inclusion of papers in the ISI "year", and this introduces a bias 
against distant and foreign publications. That is, some 1976 papers would actually be in 1977 
tapes, and more of these would be expected from abroad. We found addresses for 317,895 of the 
391,479 relevant papers. Searching for addresses after doing the journal and paper evaluations 
greatly reduces the complexity of location dictionaries. This is what we refer to as a "journal- 
driven" strategy. 


One can examine some of the available assumptions in Jonathan Cole, Stephen Cole, Social Strati- 
fication in Science, University of Chicago Press (Chicago, 1973), Chapter 8. 


J. Hughes, "Isolation of an Endogenous Compound from Brain with Pharmacological Properties 
Similar to Morphine", Brain Research, 88:2 (1975), p. 295. 


L.P. Renaud, J.B. Martin, P. Brazeau, "Depressant action of TRH, LH-RH and Somatostatin on Acti- 
vity of Central Neurons", Nature, 255 (May 15, 1975), p. 233. 
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"Immunoreactive LH-RH Neurons on the Hypothalamus Identified by Light and Fluorescent Micros- 
copy", Cell and Tissue Research, 157 (1975), p. 423, by D.V. Naik, Dept. of Anatomy, Sherbrooke; 
the MRC group at Laval produced No. 11: (Cell Tiss. Re. 161 (1975), p. 385), along with two 
endocrinologists from Tulane. 


In view of this problem, a more reasonable basis of comparison among organizations might be to 
simply compare their rank in the top percentile and their rank in the top decile. 


Y.4. hau, Poteoe Wong, He salvo, "Determination of Tetraalkyl Lead Compounds in the Atmo- 
sphere", Journal of Chromatographic Science (1976), 14, p. 162. 


Appendix I 


BIBLIOMETRICS: HISTORIC PERSPECTIVE 


Like fresh air and sunshine, knowledge is a public good in which the property, though valuable, 
cannot be privately appropriated. Nevertheless, to become a public possession, knowledge, both new and 
old, must be coded in some form by which information about an investigator's understanding of some 
aspect of nature can be circulated and retrieved. For this reason alone, science is, at least in prin- 
ciple, open to formal analysis. In practice, we do not yet know how to analyse the subtle coding 
devices which theories and paradigms seem to be, nor even to come to terms with the means by which 
such contextual elements as tacit skills are transmitted from innovative investigators to their 
apprentices. But because discursive knowledge, which appears as a non-appropriadle public good, is 
central to the whole enterprise, priority in new discoveries must be established by public recogni- 
tion; and thus formal accounts are circulated in a public way. 


Historians have frequently noted that many of the innovations important to European science took 
place in the seventeenth century. A number of scientific societies were formed at that time, along 
with the first national laboratories, to say nothing of the most fundamental contributions to physics 
and mathematics. But important changes took place as well in the means for circulating information 
about those discoveries. The meetings of the societies were important in this respect, and indeed this 
was essentially the purpose of the "invisible college", by which Robert Boyle referred to the informal 
meetings which led to founding the Royal Society of London. But lectures and demonstrations had been 
used as far back as Archimedes (along with the publication of scientific treatises), and universities 
from medieval times had preserved and greatly extended this practice. The innovation of lasting sig- 
nificance was a transformation in the means of communicating discoveries: from the private circulation 
of scientific correspondence to its formal publication in journals. 


For transformations on the scale permitted by this innovation, sociologists reserve the terms 
"gemeinschaft-gesellschaft", an argot of untranslatable majesty. But, as with other social transforma- 
tions due to changes in communications, social and normative factors turn out to be at least as impor- 
tant as the more visible technical developments. 


Private networks of investigators had previously been keeping in touch through what now seems a 
very extensive practice of writing letters. The latter often contained detailed accounts of discov- 
eries, sometimes using a cipher or secrecy code, mixed in with personal speculations and diverse ideas 
in alchemy and philosophy. Given the extreme political and religious turmoil of the seventeenth centu- 
ry, these networks were surprisingly cosmopolitan. On an informal basis, Father P. Marin Mersenne 
enormously improved the range and effectiveness of this process by re-circulating the correspondence 
from his wide range of contacts to the others in his network. Indeed, Mersenne was instrumental in 
circulating the work of Descartes, Pascal and Galileo, among many others. 


The scientific societies helped to institutionalize and formalize this process. Perhaps the earli- 
est society was the "Accademia del Lincei", established in Rome, 1603, with Galileo among its distin- 
guished members. Its Florentine successor, the "Accademia del Cimento", devoted itself more directly 
to experiments, publishing a collection of them in 1667 (though the exercise was not repeated on a 
regular basis). Rather informal scientific societies like these proliferated in Europe, eventually 
attaining at least nominal royal patronage in the Restoration England of 1662, and substantially so in 
France in 1666, where something like a national laboratory was established. 


The first scientific journals came out in 1665. The "Journal des Scgavans", edited in Paris by Denis 
de Sallo, appeared weekly with current news of experiments, books and technical inventions, and opin- 
ion on many controversial subjects, not only in natural philosophy, but including law, religion and 
the arts. Perhaps it's not surprising that the journal was frequently suppressed.(1) This experience 
informed the British, who were ale to establish a more durable instrument. 


The Royal Society of London exercised a great portion of political realism in its constitution and 
activities, surviving as it did, both civil and religious wars, incorporating all sides and offending 
none with a program that Robert Hooke described as follows: 


"The Business and Design of the Royal Society is: To improve the knowledge of natural things, and all 
useful Arts, Manufactures, Mechanics, Practices, Engynes and Inventions by Experiments (not meddling 
with Divinity Metaphysics, Moralls, Politicks, Grammar, Rhetoric, or Logick)."(2) 


See footnote(s) at end of Appendix I. 
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Their first Royal Charter (1662) accordingly granted "the full power and authority...to print such 
things, matters and businesses, concerning the said Society", and also "to hold Correspondence and 
Intelligence with any Strangers, whether Private Persons, or Collegiate Societies or Corporations, 
without any Interruption or Molestation whatsoever".(3) In due course, the "Philosophical Transac- 
tions" of the Royal Society began appearing on a monthly basis in March, 1665, by and large excluding 
all matters not strictly considered scientific by the Society and its secretaries.(4) 


Henry Oldenburg was appointed the first corresponding secretary of the Society. A German expatriate 
who had studied at Oxford, he acted for many years as consul in England for Lower Saxony. As secre- 
tary, he developed an extensive range of correspondents throughout Europe. The difference of this from 
the Mersenne network was that a "letter" to Oldenburg was recorded at the Society, and became a some- 
what more formal statement of the nature, timing and ownership of discoveries. Priority disputes were 
frequent then as now, and this represented an opportunity to quickly establish priority without pro- 
ducing a lengthy treatise. 


The Society could repay the labours of its two secretaries with some honour, but no money (reflect- 
ing its own relationship with Charles II, who became a member, but was never a great benefactor). 
Therefore, as a private venture, but with the approval of the Society, Oldenburg began publishing the 
"Transactions" in London, 1665. It provided a small income for himself, and for his readers: accounts 
of experiments performed before the Society, articles, and "letters" from correspondents on scientific 
matters, edited, of course, by Oldenburg. The Fellows of the Society, which included many foreign 
scientists, thereby acquired a direct means for the publication of accounts of their original work. Of 
course, many non-members were within Oldenburg's network of correspondents, and indeed, this seems to 
have been an important device for recruitment to the Society as well.(5) 


Oldenburg's creative work in institutionalizing the system of scientific correspondence brought him 
little financial return (indeed his independence and zest for foreign correspondence landed him in the 
Tower of London for two months),(6) and the Transactions survived until the present time, despite the 
efforts of his rival and successor, Robert Hooke, who temporarily managed to suppress publication. The 
journal allowed very large and impersonal markets in intellectual property to develop. Without knowing 
his readers, an investigator could place his intellectual property in the public domain, in exchange 
for which he might expect to be accorded public recognition of his priority of discovery. The letter 
and the paper, which have evolved much over the years into a highly formalized style, complete with 
references and proper acknowledgements of intellectual debts, could easily, and indeed in this centu- 
ry, have largely been replaced by more effective means of communication. But they endure and have 
increased their importance commensurate with the size of research communities because they are really 
pronouncements, or claims, quite as much as they were ever merely informative. 


In his study of priority disputes, R.K. Merton discerned the normative, constraining and highly 
moralistic behaviour surrounding the exchange and correct acknowledgement of intellectual property. 
Pushed into the extreme, this excessively individualist form of market competition can become 
"dysfunctional", one of the bad things in Merton's lexicon: 


"It can lead scientists to develop an extreme concern with recognition which is in turn the validation 
by peers of the worth of their work. Contentiousness, self-assertive claims, secretiveness lest one be 
forestalled, reporting only the data that support a hypothesis, false charges of plagiarism, even the 
occasional theft of ideas and, in rare cases, the fabrication of data, - all these have appeared in 
the history of science and can*be thought of as deviant behavior in response to a discrepancy between 
the enormous emphasis in the culture of science upon original discovery and the actual difficulty many 
scientists experience in making an original discovery."(7) 


Thus, the paper became the chief instrument for making priority claims in intellectual property, 
and having these claims evaluated by peers (see Reference 2). This meant initial refereeing at the 
level of editorial acceptance, but ultimately, the response to a paper is given by the scientific com- 
munity at large. In that forum, the ultimate value of a contribution is just the influence or impor- 
tance it acquires with respect to the work of others in all research communities. Accordingly, reco- 
gnition in terms of actual use may be delayed precisely because information is not free. It is 
obviously costly to produce and circulate, but more fundamentally, information is intellectually 
costly to consume, and directly so in proportion to its originality and novelty from the user's point 
of view. 


See footnote(s) at end of Appendix I. 
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Footnotes 


(1) 


(2) 


(3) 
(4) 


(5) 


(7) 


Some 111 volumes were published from 1665-1792, when the Journal was suppressed, along with other 
institutions and eminent scientists, during the French Revolution. Douglas McKie, "The Scientific 
Periodical from 1665 to 1798", Allan Ferguson (ed.), Natural Philosophy Through the 18th Century 
and Allied Topics, Taylor, Francis (London, 1948), p. 122. 

Robert Hooke (1663), quoted in Wolfgang van den Daele, "The Social Construction of Science", E. 
Mendelsohn, P. Weingart, R. Whitley (eds.), The Social Production of Scientific Knowledge, Reidel 
(Dordrecht mov jusaD. wl. 

Quoted in ibid., p. 29. 

R.K. Merton, ("Changing Foci of Interests in the Sciences and Technology" (1938), in Robert 
Merton, The Sociology of Science, University of Chicago Press (Chicago, 1973), Chap. 8), was the 
first to study the content of the early "Transactions". 

This point is advanced in L. Mulligan, G. Mulligan, "Reconstructing Restoration Science: Styles of 
Leadership and Social Composition of the Early Royal Society", Social Studies of Science, 11 
(1981), p. 327. A somewhat contrary view is in M. Hunter, "Reconstructing Restoration Science: 
Problems and Pitfalls in Institutional History", Social Studies of Science, 12 (1982), p. 451, 
emphasizing the role of others in recruitment. 

The warrant issued for his arrest (June 20, 1667) referred to "dangerous designs and practices". 
Pepys, the Walter Winchell of his time, thought this meant foreign correspondence. For a more 
thoughtful analysis, see Douglas McKie, "The Arrest and Imprisonment of Henry Oldenburg", Notes 
and Records of the Royal Society, 6 (1949), p. 28. His correspondence has now been collected: 
A.R. Hall, M.B. Hall (eds.), The Correspondence of Henry Oldenburg, six volumes, University of 
Wisconsin Press (Madison, 1965-1976). 

R.K. Merton, "Priorities in Scientific Discoveries" (1957), reprinted in Merton, The Sociology of 
Science, (op. cit.), p. 323. He elsewhere gave to this opiate of the intellectuals a more 
personal motivation, "insanabile scribendi cacoethes", the itch to publish. Robert Merton, On the 
Shoulders of Giants: A Shandean Postscript, Harcourt, Brace (New York, 1965), p. 83. 
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Appendix II 
THE ANALYSIS OF RELEVANCE 


In this study relevance is taken to be a relation among the objects which define an information 
source under given conditions of inquiry. From the point of view of an inquiring user, the relevance 
of a document depends on one's changing state of knowledge. That is, a given document may or may not 
be relevant, depending on where one is presently inquiring in the source. Given conditions of repeated 
inquiry, William Goffman pointed out(1) that a matrix of transition probabilities could be computed 
which would in effect, represent the transitivity relations or paths traced out on a source by inquir- 
ing users. 


In a source S consisting of n objects, which may be a set of documents, a vocabulary, a community 
of authors, etc., we may represent the usage of any information conveying object in the source as Uj. 
For example, this could be the file of all articles that cite the source object Sj; in a time period, 
and this would represent a particular evaluative usage. Thus, if a user is inquiring at S;, the proba- 
bility of his inquiring at S; is the common usage (or intersection of U; and U;) over the usage of 

: : k , a) 
S;- In other words, the propensity of S; users to inquire at Sj is: 


Pag = (Uj OV U5) /U; Gis) 


By measuring the probability that S; will be found useful to inquirers given the condition that S; was 
also useful, Goffman was able to rétain the subtlety of time dependency and learning in his indirect 
method of information retrieval. 


We here interpret the transition probability as a marginal propensity to inquire, and as such it is 
indirectly an estimate of the cost of inquiry, which we began to introduce at the conclusion of 
Appendix I. If a high proportion of S; users proceed to S-, the cost must be minimal. This means not 
only that the two source objects may be associated Seaetnen in terms of intellectual distance, but 
that a direction may be attached to the ordering relation of influence or relevance among source ob- 
jects. This is typically not a symmetric relation, and would only become so if the files Uj; and U; 
were identical. (Symmetry in this relation is one of the unwarranted assumptions in a ae-Osk Se lone: 
approach. ) 


While the transition probability overcomes many practical problems in the use of citations, and 
appears to accurately reflect the relations of influence among source objects, it suppresses the rela- 
tive importance of various linkages in the graph. This aspect of source usage may be re-introduced by 
means of Shannon's statistical measure of the contribution of each element of a source to the degree 
of organization of the whole source. The Bradford-Zipf curves in fact demonstrate such a "Matthew 
effect" over the whole source (R.K. Merton's term, but he failed to codify it); Shannon's entropy 
measure correctly builds up this effect from its constituent elements. 


Thus the relative importance of any of the n(n-1) ordered pairs in a source of n information con- 
veying objects is obtained from the statistical entropy of each pair in turn: 


A 

Where Pi¥ is the probability of all common usage in the source being focussed on the linkage of Sj and 
Sj? 

je n 
Py} = (UZAU;)/ SY (Uy AU;) (3) 

yest 
n 
where D3 Pais = a) late ine = P atic Symmetry in this case prevails. 
se A tt 


See footnote(s) at end of Appendix II. 
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For some purposes in studying the dynamics of research specialties, it will be useful to compute the 
entropy of a source S, consisting of n information conveying objects: 


Tee ec Pia)» eh (4) 


The two probability measures may be combined as a measure of the relevance on an ordered pair of 
source objects, or more specifically, its marginal relevance under given conditions of inquiry: 


alg = ey) (Pi) (5) 
n 
and rj = 2 Ti (6) 
i 
n ; 
and Rg = Sy Big CP) 


i,j = 1 
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Footnote 


(1) William Goffman, "An Indirect Method of Information Retrieval", Information Storage and Retrieval, 
4:4 (Dec. 1968), collected in Saracevic, ibid., p. 485. 
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Appendix III 
IMPLEMENTATION PROCEDURE 


The relations of relevance among source objects have certain mathematical properties which are use- 
ful for the purpose of analysing the structure of large and heterogeneous information sources. Again, 
following Goffman's "indirect method" (see Reference 25 and Appendix II, footnote 1), we design an 
algorithm which can read in ordered pairs of source objects incrementally from greatest to least rele- 
vance. Each of the n(n-1) ordered pairs is a possible arc or edge from one vertex to another in a 
directed graph of source objects. Each single vertex represents an information conveying object in the 
source. Each object is at least minimally relevant to itself (Pi; = 1), and this is a reflexive rela- 
tion which Goffman called conversance. 


A sequence of arcs from one vertex to a final vertex, with no repeated vertices, defines a communi- 
cation path on the source, with both reflexive and transitive properties. These paths through the 
source represent a “unilateral connection" or a partial ordering of the source. Thus, an object Sj 
will have an antecedent set, which is its range of influence. 


As the algorithm traces out these collective footprints through the source, it also observes 
another mathematical relation, this time with symmetric properties. When at least one path exists 
from a vertex i to another k, and at least one path exists from k back to i, we have an intercommuni- 
cation path, or closed walk; this is an equivalence relation with reflexive, transitive and symmetric 
properties. It therefore provides for a partition of the source into disjoint equivalence classes. A 
maximal set of objects with this property is an intercommunication cluster which we define as a 
"research front" of source objects. 


Because any data source will exhibit more or less seriously uneven disciplinary and geographical 
coverage of the system of journals, and because the publication rates vary among research fields 
whether or not they are well represented, it is necessary to build into the algorithm a relevance 
threshold that will be sensitive to these characteristics as they may vary from one cluster to 
another. (Other things being equal, the investigator may want to obtain a weak covering which would be 
a graph of the central core of a research front.) The depth first searching algorithm described below 
reads the relevance on arcs ordered incrementally, and at any given point a number of clusters are 
growing simultaneously. When the marginal relevance on an arc fails to augment the total relevance in 
that cluster by more than a threshold percentage (in our analysis of ISI journals, this was set at 
Te = -001), that equivalence class or cluster is considered to be saturated, its structure is re- 
corded, and it is then collapsed to a single vertex. 


Since the threshold is established relative to the strength of relevance relations among objects, 
the analysis yields classes of weakly connected clusters as it nears the tail of the distribution. 
This means that new research areas are not screened out of the analysis. The procedure is iterative, 
and can be repeated using clusters as vertices: this results in an analysis into "fields" of research 
fronts. However, the main value of the analysis is that it provides an evaluation of source objects in 
terms of their relevance to other source objects; it clusters those objects into mutually exclusive 
classes, and it orders those objects and classes into antecedent sets which show their range of 
influence. 


The analysis which has been outlined above would normally be carried out by means of operations 
which have been developed for sparsely filled matrices. However, an effective implementation need not 
be carried out by means of traditional techniques of matrix analysis. Indeed an implementation of the 
size proposed in this study cannot be carried out in this way. Instead, we must specify an approach by 
which the clustering algorithm can be performed on very large information sources, using present gene- 
ration computers. 


A comprehensive formalism for determining the cost of solving problems has been developed which 
allows one to select which method among many is most efficient, and thus most appropriate.(1) For any 
problem, we can associate a size, which is the number of characters required to write down the input 
to an algorithm for its solution. The number of steps used by the algorithm is a function of the size 
of the problem. In general, we give preference to an algorithm whose complexity increases as slowly as 
possible when the problem size is increased, and the contribution of formal analysis allows us to 
anticipate these growth rates. 


See footnote(s) at end of Appendix III. 
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A number of clustering algorithms have been discussed in the literature on bibliometrics. Most of 
these involve the analysis of small samples of literature, where the luxury of direct matrix solutions 
can be tolerated. We need not attempt to review those solutions here since they increase in complexity 
too sharply with increases in size, and cannot reasonably be implemented on large sources. 


The relative growth rate for the eigenvalue solutions used hitherto in bibliometric clustering pro- 
blems is in the order of O0(n*) for n Journals. For 4,000 ISI journals, this would require 10° times as 
much computation time as the implementation used here. Our goal is to obtain reliable estimates of the 
computation time required to solve a problem of size n. For an algorithm, we define S(n) to be a func- 
tions; when evaluated at k, S(k) is the time required by the algorithm to solve an input problem of 
size k. A brief technical note on this may be helpful (Colbourn, et al.).(2) 


Given the function S(n), we would be able to easily decide how much computation time our particular 
problem would use. Unfortunately, the exact determination of S(n) for most nontrivial algorithms is 
beyond current techniques. Nonetheless, procedures in algorithm analysis (see Aho, et al., Reference 
22), can be used to determine the growth rate of the function S(n). 


Along these lines, we Say the function S(n) is O[f(n)], [“order f(n)"], if there is an absolute 
constant c for which S(n) Scf(n). By way of example, for g(n) = 1.5n + 7logn + 300, we have g(n) = 
O(n): in other words, asymptotically, g(n) grows as a linear function of n. 


An algorithm is considered efficient if its computation time is o(nk) for some cone eS onmunat 
is, a polynomial in the size of the input. Similarly, an algorithm operating in O(n)K time is consid- 
ered more efficient than one which does not. Intuitively, this notion reflects our preference for an 
algorithm whose complexity increases as slowly as possible when the size is increased, as opposed to 
one whose solution operates in O(n)? time for a set of n journals. 


Having formulated the clustering problem in terms of the properties of a digraph, it is possible to 
program the algorithm to organize research fronts and collapse them to single vertices without 
increasing the complexity of the operation more rapidly than the size of the problem. R.E. Tarjan has 
described just such an optimal algorithm for clustering the strongly connected components in a 
digraph.(3) This approach required only one four-thousandth of the computer time required for the 
transitive closure solution of ISI journals. 


A necessary modification to the Tarjan algorithm involves reading in arcs which have been ranked by 
the relevance factor. This is to permit the threshold to be established with respect to a given clus- 
ter or research front, rather than imposing a single threshold for the whole source. Therefore, our 
implementation incorporates some techniques proposed by G.A. Cheston for incremental algorithms.(4) 
Using these techniques, we have analysed the structure of relevance relations in sources ranging up to 
20,000 information conveying objects. 


See footnote(s) at end of Appendix III. 
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Footnotes 


(1) 


(2) 


(3) 
(4) 


A description of this formalism is in A.V. Aho, J.E. Hopcroft, H.D. Ullman, The design and analy- 
Sis of computer algorithms (Addison, Wesley, 1974); a less rigorous presentation is in H.R. Lewis, 
C.H. Papadimitron, "The Efficiency of Algorithms", Scientific American, 238:1 (1978), p. 96. 

The algorithm was outlined in Charles Colbourn, Marlene Colbourn, James MacAulay, "Specification 
of Research Fronts: Methodology and Algorithms", Conference on Numerical Mathematics and Comput- 
ing, University of Manitoba (Winnipeg, 1980); and its implications in James MacAulay, "The World 
Brain Again: This Time with Feeling", World Classification Society, Psychometric Society, Proceed- 
ings of a Joint Meeting (Montreal, 1982). 

R.E. Tarjan, “Depth First Search and Linear Graph Algorithms", SIAM J. Comuting 1:2 (1972), 
pe l46. 

G.A. Cheston, "Incremental Algorithms in Graph Theory", Ph.D. Dissertation (University of Toronto, 
1976). An excellent survey of the field of graph theory and its relevance in many areas of policy 
research is Fred Roberts, Graph Theory and Its Application to Problems of Society, SIAM 
(Philadelphia, 1978). 


Appendix IV 
CLASSIFICATION OF RELEVANT JOURNALS 


This is a listing of journals in influential research fronts obtained from an analysis of the 
Journal Citation Reports for 1977. The listing adopts a Field: Front notation sequence, giving journal 
fronts in the order in which they saturated, which would also be the order of their strength of intra- 
connectedness and the order of their overall importance in the whole system. Therefore, more attention 
should be paid to the performance of laboratories in journal classes that occur early in the se- 
quence. They represent the mainstream of world research. 


The journal fronts can be resorted into fields by means of the Front to Field concordance listed 
below. Journals within each group are then listed in order of their immediate relevance in the system 
of ISI journals, and their percentage shares of the relevance of all journals in each front is given 
as well. Percentage shares may occasionally sum to more than 100 due to the fact that a few JCR jour- 
nals or other serials, marked with an asterisk (*), were found to be significant members of these 
groups but were not found on the correction tapes for 1978; i.e., they are not used in other ISI pro- 
ducts such as the Science Citation Index, which are generated in the 11-character format from the 
correction tapes. 


For each journal in this listing, the full name is given, along with the founding year and city of 
publication, as far as these could be determined from reference in Ulrich's International Periodicals 
Directory, the holdings of CISTI and the libraries of the Department of Health and Welfare. No cross- 
checks were run on these and, of course many, cities are simply the mailing points for journals which 
are essentially international in character. A single letter code identifies the languages in which 
papers and abstracts are published. Journal fronts with very few journals or little Canadian partici- 
pation, and one or two for which details were not readily at hand are listed here in 20-character 
short-form only. 


Field Front Concordance 


Front Field Front Field 
1 1 Do) 17 
72 1 60 nS: 
5 4 61 10 
4 3 62 27 
5 1 63 10 
6 1 64 28 
, 1 65 20 
8 1 66 18 
9 3 6/7 21 

10 ¥ 68 Zp, 
4 5 69 29 
\Z 2 70 30 
13 8 va By 
14 3 VE 2S 
ile 2 73 24 
16 12 74 52. 
ad 6 YD 26 
18 11 76 53 
4S 2 Wa if 
74h) 2 78 34 
21 2 79 7 
22 135 80 35 
oe 3) 81 9 
24 2 82 14 
DS) 5 83 7/ 
26 2 B84 36 
2a, 2 85 ie 
28 10 Total - 85 fronts 

29 5 36 fields 


LIST OF JOURNAL GROUPS BY RESEARCH FRONT 


Field. Name 
Front 


Core Biochemistry 
Core Medicine 
Core Chemistry 
Core Physics 


Gastroenterology, Surgery 
Immunology and Pediatrics 
: Chemical Physics 

o10 Solid State Physics 

1 Plant Biology 

fael| 2 Secondary Biochemistry 

ho 11D Geology and Astrophysics 
3.14 Pharmacology 


1 
2 
5 
4 
5 
-6 Cell Research 
7 
8 
9 
1 
1 


Ze MND Microbiology and Macromolecular Science 


12.16 Main Law 
6.17 Main Mathematics 
iste Neurosurgery 


Mee MNGi Nutrition, Clinical Medicine and Public 


Health 
5.20 Physical Chemistry 
Lawn Ecology 
USGlZ Opthalmology 
we 728) Physics of Materials 


2 pds Secondary Biochemistry, Human Genetics 


DisiD Physical Chemistry: U.S.S.R. 
2.26 Psychiatry, Psychology 

Dey), Immunology and Allergies 
TE Z8 Main Economics 

Did Plant Biology and Soil Science 


6.30 Applied Mathematics, Secondary Physical 


Applications 


554 Secondary Geophysics and Space Science 
Lede Secondary Microbiology, Veterinary 


Science 
16.33 Secondary Law 
7.34 Chemical Engineering 
[Soe DENELSELY 
2.36 General Clinical Medicine 
ISS Laryngology and Plastic Surgery 
10.38 Animal Biology 


2.39 Applied Microbiology, Biotechnology 


2.40 Behavioural Sciences 
Told Photo-chemistry, Applied Physics 


O42. Environmental Sciences and Chemical 


Analysis 


Pharmacology and Clinical Chemistry 


paella 


Front 


Name 


Clinical Pharmacology, Geriatrics 

Anaethesiology 

Secondary Psychiatry and Psychology 

Clinical Medicine, Reproductive Biology 

Applied Mathematics and Computer 
Serence 

Secondary Geophysics, Atmospheric 
Sciences 

Secondary Plant Biology 

Applied Statistics, Population Biology 

Secondary Chemical Engineering 

Secondary Law Group 2 

Secondary Applied Physics 

Marine Biology 

Applied Economics 

Biomedical Engineering, Respiratory 
Diseases 

Speech and Hearing 

peripheral 

peripheral 

peripheral 

Entomology 

peripheral 

Horticulture 

peripheral 

Mental Retardation 

India: Biology 

Japan: Biomedicine 

Secondary Veterinary Science 

peripheral 

peripheral 

peripheral 

peripheral Biochemistry 

peripheral 

peripheral 

peripheral 

peripheral 

peripheral 

peripheral 

Materials Science, Metallurgy 

peripheral 

Secondary Geology 

Secondary Probability 

Russia: Peripheral Chemistry 

peripheral 

Textiles 


69 


LIST OF JOURNALS IN EACH FRONT 


R.F. 


46 


ile 


as 


Wh 


11 


12 


Core Biochemistry 


NATURE, 1869, London, England, 14.4%, E. 
SCIENCE, P1885, sWashaig ams .U.ciyuulis sls) Es. 
JOURNAL OF BIOLOGICAL CHEMISTRY, 1905, 
Bethesda, Md., 12.0%, E. 

NATIONAL ACADEMY OF SCIENCES, 1915, Washington, 
Dello 5 WNsGii5 1E5 

BIOCHIMICA ET BIOPHYSICA ACTA, 1947, Amsterdam, 
Netherlands, 10.3%, E. 

BIOCHEMICAL JOURNAL, 1906, Colchester, England, 
8.2%, E. 

BIOCHEMICAL AND BIOPHYSICAL RESEARCH COMMUNICA- 
TLONSS 9595S NewsYork, USS sAs 70%, Es 
BIOCHEMISTRY (U.S.), 1962, Washington, D.C., 
Gedy Ee 

ARCHIVES OF BIOCHEMISTRY AND BIOPHYSICS, 1942, 
New York, USSSA. , 5.6705 0B. 

EUROPEAN JOURNAL OF BIOCHEMISTRY, 1967, Berlin, 
West Germany, 4.2%, E, F, G. 

JOURNAL OF MOLECULAR BIOLOGY, 1959, London, 
England and New York, U.S.A., 4.1%, E, F, G. 
FEBS LETTERS, 1968, Amsterdam, Netherlands, 
“St dE IP Ge 


Core Medicine 


LANCET, 1823, London, England, 7.9%, E. 

NEW ENGLAND JOURNAL OF MEDICINE, 1828, 7.4%, E. 
JOURNAL OF CLINICAL INVESTIGATION, 1924, New 
Works WeSoMog Golhtig (Es 

ANNALS OF NEW YORK ACADEMY OF SCIENCE, 1977, 
New YarlcomiU< SioAl a ere sols Exe 

SOCIETY FOR EXPERIMENTAL BIOLOGY AND MEDICINE, 
PROCEEDINGS, 1903, New York, U.S.A., 6.0%, E. 
FEDERATION PROCEEDINGS, AMERICAN SOCIETY OF 
EXPERIMENTAL BIOLOGY, 1964, New York, U.S.A., 
Bolin 1s 

JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 
(8BSyeChnicagae LUM. allah Es 

BRITISH MEDICAL JOURNAL, 1857, London, England, 
DOG 1B. 

AMERICAN JOURNAL OF PHYSIOLOGY: HEART AND 
CIRCULATORY PHYSIOLOGY, 1898, Bethesda, Md., 

Ne (syisry aie 

JOURNAL OF EXPERIMENTAL MEDICINE, 1896, New 
Worle, WiSSatlan “islOhsa 12. 

ANNALS OF INTERNAL MEDICINE, 1927, 
Philadelphia, Pa., 3.9%, E. 

AMERICAN JOURNAL OF MEDICINE, 1946, New York, 
WSisiNes Sotlig sc 

JOURNAL OF PHYSIOLOGY, 1878, Cambridge, 
England, 5.8%, E. 

ANALYTICAL BIOCHEMISTRY, 1960, New York, 
WS SiG INE 8 SS otibrin |S 

JOURNAL OF IMMUNOLOGY, 1916, Baltimore, U.S.A., 
Deditig iiss 

ENDOCRINOLOGY, 1917, Baltimore, U.S.A., 3.1%, 
Ee 
JOURNAL OF CLINICAL ENDOCRINOLOGY AND 
METABOLISM, 1941, Baltimore, U.S.A., 3.0%, E. 
ARCHIVES OF INTERNAL MEDICINE, 1908, Chicago, 
Wing Slike 15 

CANCER, 1948, Philadelphia, Pa., 3.0%, E. 
CIRCULATION, 1950, Dallas, Texas, 2.7%, E. 
CIRCULATION RESEARCH, 1953, Dallas, Texas, 
BANG Nix 

METHOD IN ENZYMOLOGY,* 1955, New York, U.S.A., 
VPS Se oe 

BRAIN RESEARCH, 1966, Amsterdam, Netherlands, 
ZI cole ke 

AMERICAN JOURNAL OF CARDIOLOGY, 1958, New York, 
We Sine Wainy ec 

AMERICAN HEART JOURNAL, 1925, St. Louis, Mo., 
lieGro saeee 


R.F. 


Aba 78 


5s 


4: 


12 


16 


Core Chemistry 


AMERICAN CHEMICAL SOCIETY JOURNAL, 1879, 
WASMLMGEOM gm Dialer lo zu) tere 

JOURNAL OF CHEMICAL PHYSICS, 1933, New York, 
WEScNs a WZosktin (Bo 

JOURNAL OF PHYSICAL CHEMISTRY, 1896, 
Washington, D.C., 7.8%, E. 

JOURNAL OF ORGANIC CHEMISTRY, 1936, Washington, 
Dilan Sodksg lec 

JOURNAL OF THE CHEMICAL SOCIETY, CHEMICAL 
COMMUNICATIONS, 1972, London, England, 6.1%, E. 
CANADIAN JOURNAL OF CHEMISTRY/JOURNAL CANADIEN 
DE CHIMIE, 1951, Ottawa, Canada, 6.0%, E, F. 
TETRAHEDRON LETTERS, 1959, Oxford, England, 
Bisby les lan Go 

CHEMICAL SOCIETY OF JAPAN, 
Nokyoj, dapan,, 5.5, 5 F, iG. 
CHEMISCHE BERICHTE, 1868, Deerfield Beach, 

Fla. and Weinheim, West Germany, 5.4%, G, 
summaries in E and G. 

ANGELWANDTE CHEMIE, 1888, Deerfield Beach, 

Fla. and Weinheim, West Germany, 5.0%, G, E. 
HELVETICA CHIMICA ACTA, 1918, Basel, 
Switzerland, 4.7%, G, occasionally E, F and I. 
TETRAHEDRON, 1957, Oxford, England, 4.6%, E, 

Po Bic 

INORGANIC CHEMISTRY, 1962, Washington, D.C., 
4.4%, E. 

SOCIETE CHIMIQUE DE FRANCE. BULLETIN, 1858, 
Boeing |Bo 

JUSTUS LIEBIGS ANNALEN DER CHEMIE, 1932, 
Weinheim, West Germany, 3.6%, G. 

JOURNAL OF ORGANOMETALLIC CHEMISTRY, 1963, 
Lausanne, Switzerland, 3.0%, E, F, G, Summaries 
Unive. 


BULLETIN, 1926, 


Core Physics 


PHYSICAL REVIEW LETTERS, 1958, New York, 
WeShow Wott leo 

ROYAL SOCIETY OF LONDON, PROCEEDINGS, SERIES 

A, 1665, London, England, 11.2%, E. 

JOURNAL OF APPLIED PHYSICS, 1931, New York, 
WieSctlen BoWks5 12. 

PHYSICAL REVIEW B (CONDENSED MATTER), 1893, New 
Waren, WB Soils 5 haere Wan 

PHYSICAL REVIEW A (GENERAL PHYSICS), 1893, New 
Were WsGaltliis Holic 1s 

PHYSICAL SOCIETY OF JAPAN, JOURNAL/NIHON 
BUTSURI GAKKAI OJI HOKOKU, 1946, Tokyo, Japan, 
DEO anh GM GSUMmnaries =amls) 

REVIEWS OF MODERN PHYSICS, 1929, New York, 
UoScilog Botevig 16 

PHILOSOPHICAL MAGAZINE; A JOURNAL OF 
THEORETICAL EXPERIMENTAL AND APPLIED PHYSICS OF 
CONDENSED MATTER, 1768, London, England, 5.2%, 
Ete 
PHYSICS LETTERS, SECTION A, GENERAL, ATOMIC 
AND SOLID STATE PHYSICS, 1962, Amsterdam, 
Netherlands, 5.2%, E. 

SOLID STATE COMMUNICATIONS, 1963, Elmsford, 
N.Y. and Oxford, England, 4.9%, E. 

APPLIED PHYSICS LETTERS, 1962, New York, 

USe AL 4. Gr, Ee 

JOURNAL OF PHYSICS AND CHEMISTRY OF SOLIDS, 
1956, Elmsford, N.Y. and Oxford, England, 4.8%, 
eg ing. ec 

YOURNALS Che PHYSICS Css SOLIDSSTATER PHY SLs, 
1968, New York, U.S.A., 4.0%, E. 

ANNALS OF PHYSICS, 1957, New York, U.S.A., 

Bee Ivory) Eee 

PHYSICS LETTERS, SECTION B: NUCLEAR, 
ELEMENTARY PARTICLE AND HIGH ENERGY PHYSICS, 
1962, Amsterdam, Netherlands, 2.5%, E. 


LIST OF JOURNALS IN EACH FRONT - Continued 


R.F. 


SiR 29 


lise 


16 


V7 


18 


19 


29 


Core Physics - Concluded 


PHYSICAL REVIEW, D: PARTICLES AND FIELDS, 1893, 
New York, U.S.A., 2.4%, E. 

NUCLEAR PHYSICS, SECTION A: DEVOTED TO THE 
EXPERIMENTAL AND THEORETICAL STUDY OF THE 
FUNDAMENTAL CONSTITUENTS OF MATTER AND THEIR 
INTERACTIONS, 1967, Amsterdam, Netherlands, 
Wotkss to: leg. [be 

NUCLEAR PHYSICS, SECTION B, 1967, Amsterdam, 
Netherlands, 1.5%, E. 

PHYSICAL REVIEW C: NUCLEAR PHYSICS, 1893, 
New York, USS.A., toon, Es 


Pharmacology and Clinical Chemistry 


JOURNAL OF LABORATORY AND CLINICAL MEDICINE, 
(lay Sito MowEiss= WaSioileg Wolthty, 1c 

EXPERIENTIA: MONTHLY JOURNAL OF PURE AND 
APPLIED SCIENCE, 1945, Basel, Switzerland, 
Soft Ep [Fo (Bb. Ws 

JOURNAL OF PHARMACOLOGY AND EXPERIMENTAL 
THERAPEUTICS, 1909, Baltimore, U.S.A., 5.3%, E. 
AMERICAN JOURNAL OF PATHOLOGY, 1925, 
Philadelphia, U.S.A., 4.6%, E. 

LIPE SELENCES, 19625 Oxford, England, 44%, E, 
Bg let 

BIOCHEMICAL PHARMACOLOGY, 1958, Oxford, 
England, 4.3%, E. 

ACTA PHYSIOLOGICA SCANDINAVICA, 1940, 
Stockholm, Sweden, 4.1%, E, F, G. 

AMERICAN JOURNAL OF CLINICAL PATHOLOGY, 1931, 
Philadelphia, U.S.A., 3.8%, E. 

LABORATORY INVESTIGATION: A JOURNAL OF 
EXPERIMENTAL METHODS -AND PATHOLOGY, 1952, 
Bevkesiieec, UWeSollon Gatehig tz. 

CLINICA CHIMICA ACTA, 1956, Amsterdam, 
Netherlands, 3.8% E, F, G. 

PHYSIOLOGICAL REVIEWS, 1921, Bethesda, Md., 
Solkig JEs 

BLOOD, 1946, New York, U.S.A., 3.6%, E. 
ARCHIVES OF PATHOLOGY AND LABORATORY MEDICINE, 
(92G5 Mni@eCeig Ws, Bultiis lec 

BRITISH JOURNAL OF PHARMACOLOGY, 1946, London, 
aneWeliel; Dotxig |se 
CLINICAL RESEARCH, 
elo lEc 

JOURNAL OF NEUROCHEMISTRY, 1956, New York, 
WESollog Binltia les 

JOURNAL OF GENERAL PHYSIOLOGY, 1918, New York, 
WS Aen oe Orc Es 

JOURNAL OF CLINICAL PATHOLOGY, 1947, London, 
England, 3.0%, E. . 

CLINICAL SCIENCE AND MOLECULAR MEDICINE, 
Baltimore, U.S.A., 3.0%. 

JOURNAL OF APPLIED PHYSIOLOGY: RESPIRATORY, 
ENVIRONMENTAL AND EXERCISE PHYSIOLOGY, 1948, 
Bethesda, Md., 2.9%, E. 

PHARMACOLOGICAL REVIEWS, 1949, Baltimore, 

UR Sillad stig les 

BRITISH JOURNAL OF HAEMATOLOGY, 1955, Oxford, 
England, 2.5%, E. 

PFLUEGERS ARCHIV; EUROPEAN JOURNAL OF 
PHYSIOLOGY, 1868, Berlin, West Germany, 3.3%, 
mainly E, also F, G. 

CLINICAL CHEMISTRY, 1955, Winston-Salem, N.C., 
Zn, Ee 

EUROPEAN JOURNAL OF PHARMACOLOGY, 1967, 
Amsterdam, Netherlands, 2.3%, E. 

MOLECULAR PHARMACOLOGY, 1965, New York, U.S.A., 
PZ iries |e 

JOURNAL OF PHARMACY AND PHARMACOLOGY, 
London, England, 2.1%, E. 
NAUNYN-SCHMIEDEBERG'S ARCHIVES OF PHARMACOLOGY, 
1873, New York, U.S.A. and Berlin, Heidelberg, 
Vienna, 1.9%, E. 

ARCHIVES INTERNATIONALES DE PHARMACODYNAMIE ET 
DE THERAPIE, 1894, Ghent, Belgium, 1.7%, E. 


1955; Inoroianes, Neier, 


1955, 


1949, 
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ig 


S(aid 


27 


28 


Cell Research 


JOURNAL OF CELL BIOLOGY, 1955, New York, 

WE SIahSig alts {ae 

CANCER RESEARCH, 1941, Baltimore, U.S.A., 7.4%, 
EB. 
EXPERIMENTAL CELL RESEARCH, 1950, New York, 
WloSichlon xo bdy, [Bo 

JOURNAL OF THE NATIONAL CANCER INST., 1940, 
Bethesda, U.S.A., 6.0%, E. 

JOURNAL OF BACTERIOLOGY, 1916, 

Washington, D.C., 4.8%, E. 

JOURNAL OF HISTOCHEMISTRY AND CYTOCHEMISTRY, 
195577 New York, UsSaAg, 4.0%, Es 

ANNUAL REVIEW OF BIOCHEMISTRY, 1932, Palo Alto, 
WSinitog Boethis lec 

JOURNAL OF LIPID RESEARCH, 1959, Bethesda, Md., 
Wesiailonesliosias [Ec 

JOURNAL OF CELLULAR PHYSIOLOGY, 1932, New York, 
WSS GiNerys Soehig. [Bo 

HOPPE-SEYLER'S ZEITSCHRIFT FUER PHYSIOLOGISCHE 
CHEMIE, 1877, Berlin, West Germany, 3.6%, E, G. 
ANATOMICAL RECORD, 1906, New York, U.S.A., 
3.6%, E. 

VERGEOGYS) 19555, .New York, WeS oA mai scg ee 
JOURNAL OF ULTRASTRUCTURE RESEARCH, 1957, 

New York, U.S.A., 3.2%, several languages. 

COLD SPRING HARBOUR LABORATORY, 1933, Cold 
Spring Harbour, N.Y., 3.2%, E. 

DEVELOPMENTAL BIOLOGY, 1959, New York, U.S.A., 
Sn (yh. lee, ig: (Go 

JOURNAL OF VIROLOGY, 1967, Washington, D.C., 
Dera Es 

JOURNAL OF BIOCHEMISTRY, 1922, Tokyo, Japan, 

2. B15 Bs 

INTERNATIONAL JOURNAL OF CANCER, 1966, Geneva, 
Switzerland, 2.8%, E, F. 

AMERICAN JOURNAL OF ANATOMY, 1901, New York, 
Weta Nees owkoy 1c 

BRITISH JOURNAL OF CANCER, 1947, London, 
England, 2.5%, E. 

JOURNAL OF GENERAL MICROBIOLOGY, 1947, 
Cambridge, England, 2.5%, E. 

CELL, 1974, Cambridge, Mass., 2.4%, E. 

JOURNAL OF EXPERIMENTAL ZOOLOGY, 1904, 

New York, U.S.A., 2.4%, E. 

JOURNAL OF ANATOMY, 1866, Cambridge, England, 
Dechy Tee 

JOURNAL OF COMPARATIVE NEUROLOGY, 1891, 

New York, U.S.A., 2.0%, E. 

EXPERIMENTAL NEUROLOGY, 1959, New York, U.S.A., 
PROS, |B 

JOURNAL OF NEUROPHYSIOLOGY, 1938, New York, 
WeSos5 Baltes \Ec 

EXPERIMENTAL BRAIN RESEARCH/EXPERIMENTAT ION 
CEREBRALE/EXPERIMENTELLE HIRNFORSCHUNG, 1966, 
News Yorient WaSie Alseumdiers ase Emr gas 


Gastroenterology, Surgery 


GASTROENTEROLOGY, 1943, New York, U.S.A., 

ill Seay wt es 

ANNALS OF SURGERY, 1885, Philadelphia, Pa., 

il Q/ss5 pos ee 

SURGERY, GYNECOLOGY & OBSTETRICS (WITH 
INTERNATIONAL ABSTRACTS OF SURGERY), 1905, 
Chicago, Ill., 9.4%, E. 

SURGERY emo sore MEO Us IMO tyne eiZz0lg wits 
ARCHIVES OF SURGERY, 1920, Chicago, Ill., 8.4%, 
is 
RADIOLOGY, 1923, Easton, Pa., 7.4%, E. 
AMERICAN JOURNAL OF SURGERY, 1890, New York, 
WaiaiNon We bdig 1c 

AMERICAN JOURNAL OF ROENTGENOLOGY, 1913, 
Winston-Salem, N.C., 6.5%, E. 

GUT, 1960, London, England, 6.2%, E. 

BRITISH JOURNAL OF SURGERY, 1913, Bristol, 
England, 6.0%, E. 


LIST OF JOURNALS IN EACH FRONT - Continued 
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iB 


13 


14 


Gastroenterology, Surgery - Concluded 


JOURNAL OF THORACIC AND CARDIOVASCULAR 
SURGER Vem IS ilign OleMEOULS mm Oli aie diay iE. 
JOURNAL OF UROLOGY, 1917, Baltimore, Md., 

i Googe 

BRITISH JOURNAL OF RADIOLOGY, 1928, London, 
England, 4.0%, E. 

DIGESTIVE DISEASES AND SCIENCES: FORMERLY 
AMERICAN JOURNAL OF DIGESTIVE DISEASES, 1934, 
Nene tenes WAS Na a Saein lar 


Immunology and Pediatrics 


JOURNAL OF PEDIATRICS, 1932, St. Louis, Mo., 
Da Pri Ec 

PEDIATRIES,, 1943. Evanston, DI oe2, E. 
JOURNAL OF INFECTIOUS DISEASES, 1904, Chicago, 
Is 5 Mega TEs 

AMERICAN JOURNAL OF OBSTETRICS AND GYNECOLOGY, 
(PRU Sieg leiies (Mek ayaa ea 

SCANDINAVIAN JOURNAL OF CLINICAL & LABORATORY 
INVESTIGATION, 1949, Oxford, England, 4.1%, E. 
ACTA MEDICA SCANDINAVICA, 1919, Stockholm, 
Sweden, 4.0%, E. 

AMERICAN JOURNAL OF DISEASES OF CHILDREN, 1911, 
Ce Gia, WINGS Bathe 1A 

AMERICAN JOURNAL OF THE MEDICAL SCIENCES, 1827, 
Thorofare, N.J., 3.9%, Summaries in 
interlingua. 

CLINICAL AND EXPERIMENTAL IMMUNOLOGY, 1966, 
Oxford, England, 3.8%, E. 

CANADIAN MEDICAL ASSOCIATION JOURNAL/ 
ASSOCIATION MEDICALE CANADIENNE JOURNAL, 1911, 
Ottawa, Canada, 3./%, E, F. 

ACTA ENDOCRINOLOGICA, 1948, Copenhagen, 
Denmark, 3.6%, E. 

METABOLISM, 1952, Baltimore, U.S.A., 3.6%, E. 
DIABETES.) 19>Z5eNEWE YOLK, a UnSaiAstgme edo, Ee 
ROYAL SOCIETY OF MEDICINE, PROCEEDINGS, 1907, 
London, England, 3.4%, E. 

JOURNAL OF ENDOCRINOLOGY, 1939, Colchester, 
England, 3.4%, E. 

IMMUNOLOGY, 1958, Oxford, England, 3.2%, E. 
MEDIGINE o2 22m Balt imoresn Ws SsiActsmoe Oro) Ev. 
AMERICAN REVIEW OF RESPIRATORY DISEASE, 1917, 
New) Yorles USSSA, moe Osa5) EB. 

INTERNATIONAL ARCHIVES OF ALLERGY AND APPLIED 
IMMUNOLOGY, 1950, Basel, Switzerland, 2.5%, E. 
MAYO CLINIC PROCEEDINGS, 1926, Rochester, 
NMAe Botta [is 

INFECTION AND IMMUNITY, 1970, 

Washington, D.C., 2.4%, E. 

IMMUNOCHEMISTRY, 1964, Elmsford, N.Y. and 
Obaroeels Wnelletacl, ola, [en Ir5 (6, Sire 

ARCHIVES OF DISEASES IN CHILDHOOD, 1926, 
London, England, 2.3%, E. 

QUARTERLY JOURNAL OF MEDICINE, 1907, London, 
England, 92.3%, Es 

TRANSPLANTATION, 1963, Baltimore, U.S.A., 
ain [EG 

less Ws siq Perel [Risley Ws. ollie 1s 
CELLULAR IMMUNOLOGY, 1970, New York, U.S.A., 
2.0%, E. 

ANTIMICROBIAL AGENTS AND CHEMOTHERAPY, 1972, 
Washington, D.C., 1.9%, E. 

BRITISH HEART JOURNAL, 1939, London, England, 
18%, 1B. 

OBSTETRICS AND GYNECOLOGY, 1953, New York, 
WES 9) sthhsen | Bc 

JOURNAL OF REPRODUCTION AND FERTILITY, 1960, 
Colchester, England, 1.7%, E. 

EUROPEAN JOURNAL OF IMMUNOLOGY, 1971, Weinheim, 
West Germany and Deerfield Beach, Fla., 1.5%, 
Be 
TRANSPLANTATION PROCEEDINGS, 1969, New York, 
WeSehles Wests: 1s 
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BIOLOGY OF REPRODUCTION, 1969, Champaign, Ill., 
0.9%, E. 


Chemical Physics 


CHEMICAL PHYSICS LETTERS, 1967, Amsterdam, 
Netherlands, 8.8%, E. 

CHEMICAL REVIEWS, 1924, Washington, D.C., 8.2%, 
Es 
ACCOUNTS OF CHEMICAL RESEARCH, 1968, 
Washington, D.C., 6.1%, E. 

ACTA CRYSTALLOGRAPHICA. SECTION B: STRUCTURAL 
CRYSTALLOGRAPHY AND CRYSTAL CHEMISTRY, 1948, 
Copenhagen, Denmark, 6.0%, E. 

AUSTRALIAN JOURNAL OF CHEMISTRY, 1948, 
Melbourne, Australia, 5.9%, E. 

MOLECULAR PHYSICS, 1958, London, England, 5.5%, 
ele, ee 

JOURNAL OF INORGANIC AND NUCLEAR CHEMISTRY, 
1955, Oxford, England, 5.3%, E. 

RECUEIL DES TRAVAUX CHIMIQUES DES PAYS-BAS, 
1882, The Hague, Netherlands, 5.0%, E, F, G. 
CHEMISTRY AND INDUSTRY, 1923, London, England, 
AveB raisin Ere 

ZETTSCHRIFT FUER ANORGANISCHE UND ALLGEMEINE 
CHEMIE, 1892, Leipzig, East Germany, 4.7%, G. 
COLLECTION OF CZECHOSLOVAK CHEMICAL 
COMMUNICATIONS, 1929, Prague, Czechoslovakia, 
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Slmenketntel;, 4oekiy [Bo liq (iq We 
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ILLINOIS JOURNAL OF MATHEMATICS, 1957, 
netinsenein Ils 5 25a leq ho Gy Mo 
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VETERINARY RECORD, 1888, London, England, 8.3%, 
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INDUSTRIAL AND ENGINEERING CHEMISTRY 
FUNDAMENTALS, 1962, Washington, D.C., 2.7%, E. 
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ADVANCES IN PHYSICAL ORGANIC CHEMISTRY, 
London, England, 2.2%, E. 
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JOURNAL OF HEAT TRANSFER, 1970, New York, 
WaSiaitos MWe ihan Ie 

INSTITUTION OF CHEMICAL ENGINEERS, 
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KLINISCHE ONKOLOGIE), 1903, Berlin, 
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DISEASES OF THE COLON AND RECTUM, 1958, 
Philadelphia, Pal, 1.5%, E. 
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Japan, 1.2%, J. 

INTERNATIONAL REVIEW OF NEUROBIOLOGY, 1959, 
New Yorks UiSiAti) WerZ70i5) ees 

MICROVASCULAR RESEARCH, 1968, New York, U.S.A., 
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Elmsford, N.Y. and Oxford, England, 3.1%, E. 
CURRENT TOPICS IN CELLULAR REGULATION, 1969, 
NEW Weng WoSelo 5 535 (an [Be 

BROOKHAVEN SYMPOSIA IN BIOLOGY, 1952, 
Springfield; Va., 2.9%, E. 

ADVANCES IN CARBOHYDRATE CHEMISTRY AND 
BLOCHEMISIRYspmicen2.5) New Vorkgull.S Avs 24a Ee 
JOURNAL OF APPLIED BACTERIOLOGY, 1938, London, 
England, 2.4%, E. 
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1888, Washington, D.C., 4.3%, E. 

POULTRY "SGIENCE, 1908.) Stravton, Texas, 4.27, 
E. 
PUBLIC HEALTH REPORTS, 1878, Rockville, Md., 
Diora ERs 

WIENER KLINISCHE WOCHENSCHRIFT, 1887, Vienna, 
Austria, 3./%, G. 

STROKE: A JOURNAL OF CEREBRAL 
1970, Dallas, Texas, 3.5%, E. 
PSYCHOMETRIKA, 1936, Chicago, Ill., 3.4%, E. 

BRITISH JOURNAL OF PREVENTIVE AND SOCIAL 

MEDICINE, 1947, London, England, 3.3%, E. 

BIOMETRIKA, 1901, London, England, 3.2%, E. 

ADVANCES IN LIPID RESEARCH, 1963, New York, 

UsScileg Botta tn 

ARCHIV FUER PSYCHIATRIE UND NERVENKRANKHEILTEN, 

1868, Berlin, West Germany and New York, 

Wistioile 5 74aUrso 1&5 "Bic 

ENZYME, 1961, Basel, Switzerland, 2.6%, E. 

NUTRITION SOCIETY, PROCEEDINGS, 1941, London, 

England, 2.5%, E. 

JOURNAL OF COUNSELLING PSYCHOLOGY, 1954, 

Wasiinguon. 1 DsG 4022 gk. 

EUROPEAN NEUROLOGY, 1968, Basel, Switzerland, 

Agi atin 2c 

DEVELOPMENTAL MEDICINE AND CHILD NEUROLOGY, 

1958, London, England, 2.1%, E, summaries in F, 

Gy ps 

ACTA ORTHOPAEDICA SCANDINAVICA, 1930, 

Copenhagen, Denmark, 1.9%, E. es 
LANGENBECK'S ARCHIV FUER CHIRURGIE, 1860, 

Berlin, West Germany, 1.8%, E. 

MONATSSCHRIFT FUER KINDERHEILKUNDE, 1903, 

Berlin, West Germany and New York, U.S.A., 

1.6%, summaries in E, G. 

TECHNOMETRICS, 1959, Washington, D.C., 1.5%, E. 

JOURNAL OF MEDICAL EDUCATION, 1926, Washington, 

Delos, Wolitin lec 

CHIRURG, 1928, Berlin, West Germany and 

New York, U.S.A., 1.4%, E. 

ADVANCES IN AGRONOMY, 1949, New York, U.S.A., 

Watog) Ee 

NERVENARZT, 1928, Berlin, West Germany and 

New York, U.S.A., 1.5%, E. 

ROYAL STATISTICAL SOCIETY, JOURNAL, SERIES B: 

METHODOLOGICAL, 1934, London, England, 1.3%, E. 

NUTRITION AND METABOLISM, 1959, Basel, 
Switzerland se VeZy Ee « 

JOURNAL OF HEALTH AND SOCIAL BEHAVIOUR, 
Washington, D.C., 1.2%, E. 

INTERNIST, 1960, New York, U.S.A., 1.1%, E. 
MEDICAL CARE, 1967, Washington, D.C., 1.1%, E. 
ACTA NEUROCHIRUGICA, 1950, New York, U.S.A., 
1.0%, E. 
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CANADIAN JOURNAL OF ANIMAL SCIENCE, 1921, 
Ottawa, Canada, 0.9%, E, F. 

DEMOGRAPHY, 1964, Washington, D.C., 0.9%, E. 
NUTRITION REPORTS INTERNATIONAL, 1970, 

Bos mA bos C alii) OO ra) wi. 

JOURNAL OF BIOMECHANICS, 1968, Oxford, England, 
0.8%, E. 

MILBANK MEMORIAL FUND QUARTERLY: HEALTH AND 
SOGIERY  oi9Z5ijeNew Vorkye Ui. SmiAs Oar, E-. 
PERSONNEL AND GUIDANCE JOURNAL, 1922, 
Washington, D.C., 0.8%, E. 

ANIMAL PRODUCTION, 1959, Edinburgh, Scotland, 
Wo Wig lee 

CANADIAN JOURNAL OF SOIL SCIENCES, 1921, 
Ottawa, Canada, 0.7%, E, F. 

NEUROPAEDIATRIE, 1969, Stuttgart, West Germany, 
0.7%. 

CLAYS AND CLAY MINERALS, 1870, Elmsford, N.Y. 
and Oxford, England, 0.6%, E. 

POPULATION STUDIES, 1947, London, England, 
O67, Es 

NEW ZEALAND JOURNAL OF AGRICULTURAL RESEARCH, 
1958, Auckland, New Zealand, 0.6%, E. 

BRITISH GRASSLAND SOCIETY, JOURNAL, 1946, 
Maidenhead, England, 0.5%, E. 

HELVETICA CHIRURGICA ACTA, 1945, Basel, 
Switzerland, 0.5%, F, G, summaries in E. 
ZETTSCHRIFT FUER ORTHOPAEDIE UND IHRE 
GRENZGEBIETE, 1891, Stuttgart, West Germany, 
Od roy Exe Gis 

SCHWIEZER ARCHIV FUER NEUROLOGIE, 
NEUROCHIRURGIE UND PSYCHIATRIE, 1917, Zurich, 
Switzerland, 0.4%, E, F, G, I. 

BRITISH POULTRY SCIENCE, 1960, Edinburgh, 
Scotland, 0.4%, E. 

FORTSCHRITTE DER NEUROLOGIE, PSYCHIATRIE UND 
IHRER GRENZGEBIETE, 1931, Stuttgart, 

West Germany, 0.4%, G. 

ARCHIV FUER ORTHOPAEDISCHE UND UNF ALL 
CHIRURGIE, 1903, Berlin, West Germany, 

New York, U.S.A., 0.4%, G. 

AUSTRALIAN JOURNAL OF SOIL RESEARCH, 1963, 
Melbourne, Australia, 0.3%, E. 

GEODERMA, AN INTERNATIONAL JOURNAL OF SOIL 
SCIENCE, 1967, Amsterdam, Netherlands, 0.3%, E, 
Po ee 

ANNALS OF STATISTICS, 1973, Hayward, Cal., 

Wo sre 1A 


Secondary Chemical Engineering 


NIPPON KAGAKU KEISHI/CHEMICAL SOCIETY OF 

JAPAN, CHEMISTRY AND INDUSTRIAL CHEMISTRY, 
JOURNAL, 1972, Tokyo, Japan, 8.9%, J, summaries 
Mle 

ACTA CHEMICA SCANDINAVICA, SERIES B: ORGANIC 
CHEMISTRY AND BIOCHEMISTRY, 1947, Copenhagen, 
Denmark, 7.9%, E. 

JOURNAL OF APPLIED CHEMISTRY OF THE U.S.S.R., 
1950, (English translation, New York, U.S.A.), 
Sle, eA 

CHEMICA SCRIPTA, 1971, Stockholm, Sweden, 6.4%, 
E. 
CHIMICA E L'INDUSTRIA, 1919, Milan, Italy, 

Gis Sra; ase 

POLYMER JOURNAL, 1970, Tokyo, Japan, 5./%, E. 
JOURNAL OF MACROMOLECULAR SCIENCE, PART A, 
CHEMISTRY, 1967, New York, U.S.A., 5.6%, E. 
CHEMICKE LISTY/JOURNAL OF CHEMISTRY, 1907, 
Prague, Czechoslovakia, 5.4%, Czech, Slovak, 
contents page and summaries in E. 

INDUSTRIAL AND ENGINEERING CHEMISTRY PROCESS 
DESIGN AND DEVELOPMENT, 1962, Washington, D.C., 
rail rls 

REVUE ROUMAINE DE CHIMIE, 1956, Bucharest, 
Romamitarge4s9ra,, Evy obis Gy its eOD's 
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Secondary Chemical Engineering - Concluded 


POLYMER ENGINEERING AND SCIENCE, 1961, 
Greenwich, Conn., U.S.A., 4.7%, E. 

ANNALES DE CHIMIE,* 1914, Paris, France, 4.6%, 
ea dee 

JOURNAL OF MACROMOLECULAR SCIENCE, PART B, 
PHYSICS.) 19675 (New Yorks Us Smee ayn Ee 
JOURNAL OF FLUORINE CHEMISTRY, 1970, Lausanne, 
Switzerland, 4.1%, E. 

ADVANCES IN ORGANIC CHEMISTRY,* 1972, New York, 
WeiSivAlennd etlivats ae 

JOURNAL OF APPLIED CHEMISTRY AND BIOTECHNOLOGY, 
195 Oxford, Englands 9,0. 

ANGELWANDTE MAKROMOLEKULARE CHEMIE, 1967, 
Basel, Switzerland, 3.7%, G, E, F. 

RHEOLOGICA ACTA, 1958, Darmstadt, West Germany, 
Pe(eirie, |B, [Ey 

COLLOID AND POLYMER SCIENCE, 1906, Darmstadt, 
West Germany, 2.5%, E, G. 

MICROCHEMICAL JOURNAL, 1957, New York, U.S.A., 
Jo Bye Nee 

SOCIETY OF RHEOLOGY, TRANSACTIONS, 1957, 

New York, U.S.A., 2.2%, E. 

CHEMICKE ZVESTI/SLOVENSKA CHEMICKA, 1947, 
Bratislava, Czechoslovakia, 1.9%, E. 

JOURNAL OF MACROMOLECULAR SCIENCE, PART C, 
REVIEWS IN MACROMOLECULAR CHEMISTRY, 1967, 

NERD Afenslkes WheySio/Nor, techtiq TEx 

ORGANIC PREPARATIONS AND PROCEDURES 
INTERNATIONAL, THE NEW JOURNAL FOR ORGANIC 
SYNTHESIS, 1971, Highlands, Mass., 1.3%, E. 
ADVANCES IN POLYMER SCIENCE, 1958, Heidelberg, 
Valenna, “1.1%, 9G, EE, F. 


Secondary Law Group 2 


FORDHAM LAW REV 
INDIANA LAW J 
NOTRE DAME L 
RUTGERS LAWYER 

U CINCI LAW REV 
BOSTON U LAW REV 
OHIO STATE LAW J 
WASH LAW REV 

U ILLINOIS LAW FORUM 
U PITT LAW REV 
BUFFALO LAW REV 
SYRACUSE LAW REV 
AM CRIM LAW REV 
KY LAW J 

J LEGAL STUD 
OKLA LAW REV 


Secondary Applied Physics 


JOURNAL OF QUANTITATIVE SPECTROSCOPY AND 
RADIATIVE TRANSFER, 1961, Elmsford, N.Y. and 

Oy tonetal, elatelllelaelse (asin ln. lP4, (Big vito 

AUSTRALIAN JOURNAL OF PHYSICS, 1948, Melbourne, 
Australia, 9.4%, E. 

AMERICAN JOURNAL OF PHYSICS, 
WESiciNod lois. lc 

PHYSICS TODAY, 1948, New York, U.S.A., 5.5%, E. 
REVUE DE PHYSIQUE APPLIQUEE (SUPPLEMENT TO THE 
JOURNAL DE PHYSIQUE), 1966, Paris, France, 
Ofer gee bis 

JOURNAL OF COMPUTATIONAL PHYSICS, 1966, 

New York, U.S.A., 4.7%, E. 

CRYOGENICS, 1960, Guildford, England, 4.7%, E. 
PROGRESS IN SOLID STATE CHEMISTRY, 1964, 
Elmsford, N.Y. and Oxford, England, 4.5%, E. 
ANNALES DE PHYSIQUE, 1914, Paris, France, 4.3%, 
Se liic 

INDIAN JOURNAL OF PURE AND APPLIED PHYSICS, 
1963, New Delhi, India, 4.3%, E. 
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ACTA PHYSICA POLONICA (SERIES A: GENERAL 
PHYSICS weSOLIDESTATE IRPHYSICS. SAPPELTEDSPHYSIES)y 
1932, Warsaw, Poland, 4.1%, various. 

JOURNAL OF STATISTICAL PHYSICS, 1969, New York, 
WAS SA Srgnect celica (ers 

METHODS IN COMPUTATIONAL PHYSICS, 1963, 
NEWMYOT Ken UlsaisiAes ne). Gray, 8 

SPRINGER TRACTS IN MODERN PHYSICS, 1964, 

New York, U.S.A., 3.5%, E. 

JOURNAL DE PHYSIQUE LETTERS, 1872, Paris, 
France, 3.3%, summaries in E, F. 

PHYSICAL ACOUSTICS: PRINCIPLES AND METHODS, * 
TIGES INEWR YOK su Wa Se Ales) S275 Ee 

CRC CRITICAL REVIEWS IN SOLID STATE AND 
MATERIALS SCIENCES, 1970, Cleveland, Ohio, 
Sha, led 

PHYSICA: EUROPHYSICS JOURNAL, 1934, Amsterdam, 
NetnerliancdSmyZerZ2rs Ey “ins tlre 

CONTEMPORARY PHYSICS, 1959, London, England, 
aS Pay IE 

ACTA PHYSICA AUSTRAICA (E.P.S. EUROPHYSICS 
JOURNAL), 1947, Springer-Verlag (Berlin, 

New York), 1.9%. 

INDIAN JOURNAL OF PHYSICS AND PROCEEDINGS OF 
THE INDIAN ASSOCIATION FOR THE CULTIVATION OF 
SCLENC Ee 9Z6y Calicut tauelmcitalm Wacom 
ASTRONOMICAL SOCIETY OF JAPAN, PUBLICATION, 
1949, Tokyo, Japan, 1.5%, E. 

ASTRONOMICHESKII ZHURNALS, 1924, Podsosenskii, 
Uso Solon Woes, Is 

FORTSCHRITTE DER PHYSIK, 1953, Berlin, 

East Germany, 1.5%, E, G. 

OBSERVATORY, 1877, Hailsham, England, 1.0%, E. 
RIVISTA DEL NUOVO CIMENTO, 1969, Bologna, 
Dalya le @royie te 


Marine Biology 


JOURNAL OF PSYCOLOGY, 1965, Columbus, Ohio, 
othe [a5 lao (Bin Siok, SWiMeeRSs) anh [Ec 

AMERICAN MICROSCOPICAL SOCIETY, TRANSACTIONS, 
1880, Lawrence, Kansas, 8.8%, E. 

JOURNAL OF EXPERIMENTAL MARINE BIOLOGY AND 
ECOLOGY, Amsterdam, Netherlands, 8.2%, E. 
JAPANESE SOCIETY OF SCIENTIFIC FISHERIES, 
BULLETIN, 1932, Tokyo, Japan, 6.4%, E, 
summaries in European lanquages. 

ZOOLOGISCHER ANZEIGER, 1878, Jena, East 
Germany, 6.0%, E, F, G, summaries in E, G. 
INTERNATIONAL JOURNAL FOR PARASITOLOGY, 1971, 
Elmsford, N.Y. and Oxford, England, 5.4%, E. 
JOURNAL OF FISH BIOLOGY, 1969, New York, 

WE Salou. Do Btn l2 Sd 

ZEITSCHRIFT FUER PARASITENKUNDE, 1928, 

New York, U.S.A., 4.9%, E, F, G, summaries in 
ES 
FISH BULLETIN, 1935, Sacremento, Cal., 4.4%, E. 
ARCHIVES DE ZOOLOGIE EXPERIMENTALE ET GENERALE, 
1972, Paris, France, 4.2%, F. 

JOURNAL DU CONSEIL, INTERNATIONAL COUNCIL FOR 
THE EXPEGRATIONSOR THE SEAS. 1926," 5.970, =, fo 
SOCIETE ZOOLOGIQUE DE FRANCE, BULLETIN, 1976, 
Paris, (Prance’s slog Ey tty (Ge 

HELGOLANDER WISSENSCHAF TLICHE 
MEERESUNTERSUCHUNGEN (MARINE INVESTIGATIONS), 
1937, Hamburg, West Germany, 3.6%, E, F, G. 
PROGRESSIVE FISH CULTURALIST, 1938, 

Washington, D.C., 3.3%, E. 

ANNALES DES SCIENCES NATURELLES, ZOOLOGIE ET 
BIOLOGIE ANIMALE, 1824, Paris, France, 2.3%, 
summaries in E, F. 

HELMINTHOLOGICAL SOCIETY OF WASHINGTON, 
PROCEEDINGS, 1934, Lawrence, Kansas, 2.9%, E. 
PACIFIC SCIENCE, 1947, Honolulu, Hawaii, 2.7%, 
25 
JOURNAL OF HELMINTHOLOGY, 1923, London, 
England, 2.4%, E. 


LIST OF JOURNALS IN EACH FRONT - Continued 


R.F. 


10:54: 


(iasiboe 


Zeois 


12) 


20 


21 


1s 


10 


Marine Biology - Concluded 


BULLETIN OF MARINE SCIENCES, 1951, Miami, Fla., 
Bo shin Wea Sie 

AUSTRALIAN JOURNAL OF MARINE AND FRESHWATER 
RESEARCH, 1950, Melbourne, Australia, 2.2%, E. 
ADVANCES IN PARASITOLOGY, 1963, New York, 
WeSctlin Aglliig IES 

PROTISTOLOGICA, 1965, Paris, France, 1.6%, E, 
Ee 


Applied Economics 


MANAGEMENT SCIENCE, 1954, Providence, R.1., 
11.2%, E. 

HARVARD BUSINESS REVIEW, 1922, Boston, Mass., 
Wh, 1c 

OPERATIONS RESEARCH, 1952, Baltimore, Md., 
10.1%. 

JGURNAL OF FINANCE, 1946, New York, U.S.A., 
Breilicorye ee 

JOURNAL OF LAW AND ECONOMICS, 1958, Chicago, 
WINS 6 oss [a 

BELL JOURNAL OF ECONOMICS AND MANAGEMENT 
SCIENCES) 19705 New York, U.SeAn 2.07, E. 
JOURNAL OF BUSINESS, 1928, Chicago, Ill., 5.6%, 
E. 
SOUTHERN ECONOMIC JOURNAL, 1933, Chapel Hill, 
Noliog DavAiie 1c 

JOURNAL OF ECONOMIC LITERATURE, 1963, 
Nashville, Tenn., 5.0%, E. 

JOURNAL OF MARKETING RESEARCH, 1964, Chicago, 
ING “Woy 1s 

NATIONAL TAX JOURNAL, 1948, Columbus, Ohio, 

AT. Osage Ee 

OXFORD ECONOMIC PAPERS, 1938, London, England, 
Balhog (Ba 

CANADIAN JOURNAL OF ECONOMICS/REVUE CANADIENNE 
D'ECONOMIQUE, 1968, Toronto, Canada, 3.6%, E, 
fe 
JOURNAL OF MARKETING, 1936, Chicago, Ill., 
offs. Be 

JOURNAL OF MONEY, CREDIT AND BANKING, 1969, 
Columbus, Unie; 3.2%, Es 

MATHEMATICAL PROGRAMMING, 1978, Amsterdam, 
Netherlands, 2.8%, E. 

JOURNAL OF PUBLIC ECONOMICS, 1972, Amsterdam, 
Netherlands, 2.5%, E. 

BROOKINGS PAPERS ON ECONOMIC ACTIVITY, 1970, 
Washington, D.C., 2.4%, E. 

ACCOUNTING REVIEW, 1926, Sarasota, Fla., 1.9%, 
Eo 


Biomedical Engineering, Respiratory Diseases 


JOURNAL OF NUCLEAR MEDICINE, 1960, New York, 
WESIChNs Gg Wisin TEs 

AMERICAN SOCIETY FOR ARTIFICIAL INTERNAL 
ORGANS, TRANSACTIONS, 1955, Washington, D.C., 
BG Ee 

ANNALS OF CLINICAL RESEARCH, 1969, Helsinki, 
Rinlands 57s). 

INTERNATIONAL JOURNAL OF APPLIED RADIATION AND 
ISOTOPES, 1956, Elmsford, N.Y. and Oxford, 
England, 4.8%, E, F, G, R. 

ANNALS OF ALLERGY, 1943, Minneapolis, Minn., 
US Tar, 1c 

BRITISH JOURNAL OF DISEASES OF THE CHEST, 1907, 
London, England, 3.8%, E. 

JAPANESE CIRCULATION JOURNALS, 1947, Kyoto, 
Uline Badiog. dk, 12 

HEALTH PHYSICS, 1958, Elmsford, N.Y. and 
Oxford, England, 3.6%, E. 

STRAHLENTHERAPIE, ARCHIVES OF CLINICAL AND 
EXPERIMENTAL RADIOLOGY, 1912, Munich, 

West Germany, 3.3%, G, summaries in E, F, G. 
ARCHIVES DES MALADIES DU COUER ET DES 
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VAISSEAUX, 11908), Paris, France yeos50, i. 
SCANDINAVIAN JOURNAL OF RESPIRATORY DISEASES, 
1924, Copenhagen, Denmark, 3.1%, E. 

ACTA ALLERGOLOGICA, 1948, Copenhagen, Denmark, 
3.0%, E, F, G, Summaries in E. 

DANISH MEDICAL BULLETIN, 1954, Copenhagen, 
Denmark, 2.8%, Summaries in interlinqua. 
PHYSICS IN MEDICINE AND BIOLOGY, 1956, Bristol, 
Spelletitely Actes 125 14 (4 ING 

JOURNAL DE RADIOLOGIE, D'ELECTROLOGIE ET DE 
MEDECINE NUCLEAIRE, 1914, Paris, France, 2.7%, 
F, summaries in E. 

JAPANESE HEART JOURNAL, 1960, Tokyo, Japan, 
ZieGr5 Es 

LYON MEDICAL, 1869, Lyon, France, 2.4%, F. 
IEEE. TRANSACTIONS: BIOMEDICAL ENGINEERING, 
1954.) Piscataway, Nadi, 2eor, 1B. 

ANNALES DE RADIOLOGIE: RADIOLOGIE CLINIQUE, 
RADIOBIOLOGIE, 1958, Paris, France, 2.1%, E, F. 
JOURNAL DE CHIRURGIE, 1908, Paris, France, 
Baldwin Ve 

COMPUTERS AND BIOMEDICAL RESEARCH, 1966, 

Nien? \Werlke Wis'S5iNon Zdl5 1Bc 

MEDICAL AND BIOLOGICAL, ENGINEERING AND 
COMPUTING, 1963, Stevenage, England, 1.9%, E, 
ng, Bio 

JOURNAL D'UROLOGIE ET DE NEPHROLOGIE, 1912, 
Paris, France, 1.6%, F. 

ANNALES DE CHIRURGIE, 1947, Paris, France, 
oDHp [Eo 

JOURNAL OF BIOMEDICAL MATERIALS RESEARCH, 1966, 
Nem? \eusl4, WoSiciNa 5 otis. 1c 

ACTA RADIOLOGICA SERIES 2: THERAPY, PHYSICS, 
BIOLOGY, 1921, Stockholm, Sweden, 1.4%, E. 
JOURNAL OF ELECTROCARDIOLOGY, 1968, San Diego, 
CzG Wo Skis 1s 

MODERN CONCEPTS OF CARDIOVASCULAR DISEASE, 
1932, Dallas, Texas, 1.3%, E. 

RESPIRATION, 1944, Basel, Switzerland, 1.3%, E, 
F, G, summaries in F, G. 

ACTA CARDIOLOGICA, 1946, Brussels, Belgium, 
WZ cages sielhne 

LYON CHIRURGICAL, 1908, Paris, France, 
summaries in E. 

REVUE DE CHIRURGIE ORTHOPEDIQUE ET REPARATRICE 
DE L'APPAREIL MOTEUR, 1908, Paris, France, 
1.2%, F, summaries in E. 

COEUR ET MEDECINE INTERNE, 1962, Paris, France, 
Osi lip 1Bc 


1.22 ere 


Speech and Hearing 


VISION RESEARCH, 1961, Elmsford, N.Y. and 
Oxford, England, 42.0%, E, F, G, R. 

ACOUSTICAL SOCIETY OF AMERICA, JOURNAL, 1929, 
New York, U.S.A., 40.1%, E. 

JOURNAL OF SPEECH AND HEARING RESEARCH, 1958, 
Washington, D.C., 7.2%, E. 

JOURNAL OF SPEECH AND HEARING DISORDERS, 1936, 
Washington, D.C., 6.1%, E. 

ACUSTICA, 1951, Stuttgart, West Germany, 3.7%, 
ec 
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Entomology 


JOURNAL OF INVERTEBRATE PATHOLOGY, 1959, 

New York, U.S.A., 27.5%, E. 

BULLETIN OF ENTOMOLOGICAL RESEARCH, 1910, 
London, England, 24.0%, E. 

ENTOMOLOGIA EXPERIMENTALIS APPLICATA, 1958, 
Amsterdam, Netherlands, 16.3%, E, F, G, 
summaries in language of country. 

JOURNAL OF MEDICAL ENTOMOLOGY, 1964, Honolulu, 
Hawaii, 13.0%, E. 

MOSQUITO NEWS, 1940, Fresno, Cal., 12.4%, E. 
ZEITSCHRIFT FUER ANGELWANDTE ENTOMOLOGIE/ 
JOURNAL OF APPLIED ENTOMOLOGY, 1914, Hamburg, 
West Germany, 6.9%, E, F, G, abstracts in E. 


Peripheral 


NEMATOLOGICA 


Horticulture 


J AM SOC HORTIC SCI 
HORTSCIENCE 
JRBORILE SEL 


Peripheral 


BRIT J VENER DIS 


Mental Retardation 


AMERICAN JOURNAL OF MENTAL DEFICIENCY, 1940, 
Albany, U.S.A., 19.0%, E. 

EXCEPTIONAL CHILDREN, 1934, Reston, Va., 9.3%, 
Ee 

JOURNAL OF CHILD PSYCHOLOGY AND PSYCHIATRY AND 
ALLIED DISCIPLINES, 1960). 7.7%, (Es 
MERRILL-PALMER QUARTERLY OF BEHAVIOR AND 
DEVELOPMENT, 1954, Detroit, Mich., 7.6%, E. 
BRITISH JOURNAL OF EDUCATIONAL PSYCHOLOGY, 
1931, Edinburgh, Scotland, 6.7%, E. 

GENETIC PSYCHOLOGY MONOGRAPHS, 1926, 
Provincetown, Mass., 6.5%, E. 

MONOGRAPHS OF THE SOCIETY FOR RESEARCH IN CHILD 
DEVELOPMENT, 1935, Washington, D.C., 6.3%, E. 
JOURNAL OF LEARNING DISABILITIES, 1968, 
Chucago, le 5\.G cosine 

HUMAN DEVELOPMENT, 1958, Basel, Switzerland, 
Wdlevtin, 1c . 

AMERICAN ACADEMY OF CHILD PSYCHIATRY, JOURNAL, 
1962, New Haven, Conn., 4.7%, E. 

PHI DELTA KAPPAN, 1915, Bloomington, 
4.0%, E. 

PSYCHOLOGY IN THE SCHOOLS, 1964, Brandon, Vt., 
Dieuiizaig ee 

EDUCATIONAL RESEARCH, 1958, Oxford, England, 
eihin Vo 


Indiana, 


India: Biology 


CURR SCI INDIA 

INDIAN MED RES 
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INDIAN J EXP BIOL 


Japan: Biomedicine 


JPN J EXP MED 
JPN J MED SCI BIOL 
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Secondary Veterinary Science 


LABORATORY ANIMAL SCIENCE, 1950, Joliet, 
Siete Ee 

ACTA VETERINARIA SCANDINAVICA, 1959, 
Copenhagen, Denmark, 14.5%, E, summaries in 
Danish, E. 

VETERINARY PATHOLOGY, 1964, Baltimore, Md., 
VAdeymley (Gc 

CANADIAN VETERINARY JOURNAL/REVUE VETERINAIRE 
CANADIENNE, 1960, Ottawa, Canada, 8.0%, E. 
NORDISK VETERIMAEAMEDICIN/SCANDINAVIAN JOURNAL 
OF VETERINARY SCIENCE, 1949, Copenhagen, 
Denmark, 7.8%, Danish, E, Norwegian, Swedish. 
VETERINARY MEDICINE/SMALL ANIMAL CLINICIAN, 
1905, Bonner Springs, Kansas, 7.5%, E. 

JOURNAL OF SMALL ANIMAL PRACTICE, 1960, Oxford, 
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1881, New York, U.S.A., 8.8%, E. 

TEXAS REPORTS ON BIOLOGY AND MEDICINE, 1943, 
Galveston, Texas, 7.9%, E. 

HARVEY LECTURES, 1953, New York, U.S.A., 7.0%, 
ES 

MEDICAL BIOLOGY, 1974, Helsinki, Finland, 5.9%, 
ES 
JOURNAL OF CYCLIC NUCLEOTIDE RESEARCH, 1975, 
New York, U.S.A., 5.3%, E. 

BIOFIZIKA, 1956, Moscow, U.S.S.R., 4.7%, R, 
summaries in E. 
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Elmsford, N.Y. and Oxford, England, 3.9%, E. 
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STUDIA BIOPHYSICA, 1966, Berlin, East Germany, 
eh AC 
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INTERNATIONAL JOURNAL OF PEPTIDE AND PROTEIN 
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England, 2.5%, E. 
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contents page in E. 
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CHEMICAL GEOLOGY, 1966, Amsterdam, Netherlands, 
Woe lq eo Bo 
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SEDIMENTOLOGY, 1952, Oxford, England, 6.7%, E, 
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TABLE 1. International Shares of Relevance for Peripheral Medical Research Fronts, 
1978 Reference Year 
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TABLE 1. International Shares of Relevance for Peripheral Medical Research Fronts, 


1978 Reference Year - Continued 
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TABLE 1. International Shares of Relevance for Peripheral Medical Research Fronts, 
1978 Reference Year - Continued 


Share of Share of Share of Share of top 
Research front/country relevant relevance top’ decile percentile 
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per cent 
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TABLE 1. International Shares of Relevance for Peripheral Medical Research Fronts, 
1978 Reference Year - Concluded 


Share of Share of Share of Share of top 
Research front/country relevant relevance top decile percentile 
papers(1) relevance relevance 
per cent 
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DEMO e Mars ais cia oss elem 6 ss 0.8 600 sia wiShetetayele Gvalereta o's ee 6.4 ical (oe 
WN ESU GELMANY ces ca ccccs asc cscs te ce ese we see 8.2 8.4 8.6 113551 
Pee SWEU Clit ate oye e)c's aist9.o\ 6 5)'s1 0 s'sie e's 16 0,6) l018) Fe Sie 0 oie s Daa hee 226 Fad 
lab WETIEO RV Sad 6 OO COCO OOOO OO Od OO SOC DCG emer 51 ae 2.9 [ay 
7 GEE) cocosbooooHsbonoUODOUoO OOOO O OCOD GDC Doll Lae ice ae 
BMG LUNA Kigtsts iets, sipio16's e110) c16. 6:5. © eieter crete ete iets cots Zed Lew 7a 6.9 
ny SURE RS "We GS AES cen aan Soe BIRD Gotaewl OOo Hae Zeal 1.4 alee == 
i) Ses Me hie ie Bing on ae omtcein Orhan Soa cleieoe eltie eles' Dre hoe, Aad, a 
ih OM Ya els 5 AS GAGES Baas Com tans t ar nico Zend) VS6 0.8 =~ 
TZ RINCR NOTH ANUSay «i sale iss sie ove os 6) sv Sort: GRA Gan Ze $18) 4.0 NS) 
(Ns ESSE Gece saiad ARe Como u ooo 6 OCHS 1.8 Aye ied -- 
SEs tite Rie bist oie ose ete aap: ele eue:e 6 evavec 0 61610) 00's eho 0: 0010, 1.8 1.8 A\nexd -- 
Hep POCORN! Uicsus: acne Stereo eslexe-eie oleles ss 5 06 cc cee vces 128 ea, |e) Pays 
M6; AUStTalL ane sss sc cccss AOS aor SOOO 67 2.9 jp) Gal 
ihe? 0) Fal iheAN Clidvareie c.o.0e a ata pietetslalelais)crets iatels) lal sus visteie's 16 159 Drath Ba 
18 Czechoslovakia .......e66 Siafeleinisters so sib. ptejeis siece Le ilar et) -- 
io OW ZeT andins sles ete cies cles 6 AEG Gg Rocko RO OIE ee died a8) a= 
20 East Germany ..-.ccceces BAL Go Bently 2 eer V2 1 ae == 


(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate 
tapes, and those countries with at least 1% of fractionally allocated addresses are shown. 
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TABLE 2. International Shares of Relevance in Applied Mathematics, Secondary Physical Applications (R.F.6:30), 
1978 Reference Year 


Share of Share of Share of Share of top 
Country relevant relevance top decile percentile 
papers(1) relevance relevance 
per cent 
i) Withttssl SGEUES conmconoonomounojoUpDsSaSucuONOSKGO0D 47.6 56.1 60.6 82.4 
2 laiveiltatitel Goncoocouqoono oon aor aielelie(o) alaielelecolotelstelerelavenal sie 90 voll 8.8 5.8 
De DAMCClereyeteleletederchererefetel otal etetelsetetatoletenevelsistcleieteneteterststen-tsteters Ud ee. Vel Boh 
A WES CMG CLM Muctaletetaletetstetatelotatelciatoteteleleteletotalotatetsistatotal staietere Vat N52) 4.6 -- 
Di MMIADAMMencl eierevelcvelelereteretsreiersistetetersietercicverelalelotonciorote ROOD OO Or (At, 15) Dsl 2.6 -- 
@ eMetee soootoonannc leveled sel sliekelal lclellotelel sieietetslstoichstetelcielcrs . 3.4 DES) 1.4 -- 
7 ieails, coosnwe slevoteVeferonstcl slelotelal svotoleValeVelole) lsh iolelaioisietstohstelers 1.8 Be De 7.0 
a) Walia) cooanpandoncugaccodoogoocouDbON SogndoosgGe6 : led 0.9 0.4 -- 
SMPLSING Cllleetoretatetel stools! ofeliote) sfelcrelereveneisie: ekoletelskereneiautere Seateneratets di ilar a? -- 
Mel WeScSaltle cosas Soin bc K ado oRObaSGoHGOONOOS soo agonDOage 1.6 Vas ile 0.8 
lulING thems aiicl Surertcetarcteloreletelsisteteletcterol eel latelletercteefoieke) sfetercverst= il 11633 ile 0.4 
2 INS elie, Hood gaesnoulc pilots otisia stolatelelstel ej steratctoicleleketeteraasests lo® 1.4 Wo 8) oS 
13> Switzerland ye oonAfden6 ode etistietelelelelereraicle 5000 006¢ oe: O29 0.5 == 
A Wslalinetele Ava gosonoooocondahudocbaseodouKK eV sioielie/sfeiele:e ee: ileal O29 -- 
I> Swesein cucconcamasoononoo oan ad ano DUe OOD HONDO OOOO flea 0.8 0.6 -- 


(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate tapes, and those coun- 
tries with at least 1% of fractionally allocated addresses are shown. 


TABLE 3. International Shares of Relevance in Applied Engineering Research Fronts, 1978 Reference Year 


Share of Share of Share of Share of top 
Research front/country relevant relevance top decile percentile 
papers(1) relevance relevance 
per cent 
Chemical engineering (R.F.7:34) 
i) WihttsGe) SEeueS cadsoneodouuwoosooséoanudugo ob ouudds 26.6 Pays S55) 24.0 
f2 ABE). co VOODOO OOUDOOODODO0 OU DDOOOOUOUGOODON GO OUGIC J) || 8.8 8.7 6.6 
By nahi: nonsadouloOODUnOOUb UDO DOU DOOD GAnCCOOAONO000 ° 9.0 9.4 10.5 Woe 
A WesitmGenmamyi verre steers SHoOnOnOSaODS DOO MOCORn OOOO G0 8.1 8.5 8.3 8.8 
> ROilanclesicrsiswiei«c sels) sre svellelaisieleisiclele siejever sie wisheisiehersieisiefelecers 6.9 226 563 4.6 
6 (East Germany ws... sislotalerateleteisvevelelstelsiererele seno005068 6.5 O5Z Salt Bod 
felines aocodoood cod Biolisfosalereleie/ si siereys tee e ec cesaeccvcve Big 6.4 Dew) 4.1 
SP Enailam clibaretere stercrereislorstererercrers erotelcnetsnsyeienerersteis O00 6000.5 010 4.7 BoD} Dia 6.0 
be) Whey Cosgomoulgs0N06 CeO D0G OdoODn OC srarelelenoiers) sieve ela eceieie 3.4 3.4 4.1 2.0 
(0) [sWinteeiey) nencpoDonGs Osh HocO DON CUOUG OOO OCU Ob S900000 N0 Bn) Bee Dar} Zea 
ple OW ite ZencllcimGllmrelerstaleletelotelete syeteietelelalelctel oslo siolsisinteleersioleisiorsys Zell Daal fn 5 oT 
il Zeus CCH UCI MraMete tons neretetets tetotereteretalctstaretovererereis tetera tetarerstorererevenecrons od 1.8 ee -- 
ID AM CE re steteleleleteieislsy srererelereleler Verse) clsteteloustotatelalcrelerols sloretetels tad 2.0 2.0 2.0 
(  Neeeell osoonocagacadnona0ccnas iste) stall s/sisVotatoels) sis 500 608 sD ile Wel -- 
(ies WEG pesococ sietalolopelerstctckaledetelstetetetoaetereteisteistaretctetctoreicte a6 1.6 535 dod -- 
liGmINGEei Macs cleratetalalslelsisie ctets 5 siseee Bodoaossue thle 3 lve 2 lez -- 
a AUS ter allie Bre refer eyev=)siojelersiels hel ouetavelel slelelors  aielersislerelcrevevelcveistera De9) Je) 4.2 WleZ 
Photo-chemistry, applied physics (R.F.7:41) 
i Whirl Seeley HogeoococononoucGoHe HOGSHOOS FOU GOUEtE 36.5 Boe S35) 41.4 
je \ee ESRI cooocococedd osielee ele shelarelotstel sicielateretere ies 5c 14.4 16.7 Ws 19.6 
Sh Heimeltelnel moaiacssucadooc noo Sosgnaloonono po ougnosonss Wail 8.7 Sho) ral 
VES TATA CMots eveie sh ctelels) vishe) sisves) Neher se sececeens syelieleheielllsiisliviels 6.8 3.0 2.0 -- 
Diet G POM Maticte/ovatorsieiscave sie)eleuslaieteleve wie) ovens [e) 6) 6 skels ever ctetetercielereior Die 1.6 0.8 -- 
i. Wednreriaitekey saconcoshoapdacacooododne EGO OAGTOUO DOSS Bol 4.5 Dink 1.6 
PP VIGNE soc toep GARB OmOCOD ae SCOUUnOOOCUn OD ao OtoT 6 ° So Die) Dial, 6.1 
Se Weis siiktal *ediooctwo sa ObadedsondobdsouguctodcGorDocdd - ire Zl. 209 6.5 
Fim INNGIEEE steters: feo) sal ator o e\ chelate etetet tee Slekaleleile svsielele atatetofelievelisterers 2.0 tho! 162 he 
ORE Seititez: CNM AAG vere) <r otele! eal ol etei steele! ole¥e) slakeiotarel sholeteldletolstetere¥evolsle thaw! 1.0 0.8 -- 
(ti) AbEGIY Sas dnaGouododnsd6o o0 004 fe) sbavolatel ala ioieishaletey<Kelsielsieis ° 1.6 ile les -- 
UZ PO cai\ Cl Mais rc) areie, sjcioveistsioliclsietsl stein oiorelolelolsietete atelsleielersic sistciele 1.3 1.8 Tse: 4.3 
MUS EAC More rene 1c wisi! © iovoleleleielelerohetelcvatotetel etete atetere/creuetetatelersiets tho az iG ilies 
(ete AIG eral Ne ctetensronchenelswera Peo COD UO ODOUTONIOm OOO Mot Cc sieht oi ila 3.8 4.5 4.8 


(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate tapes, and those count- 


ries with at least 1% of fractionally allocated addresses are shown. 
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TABLE 4. International Shares of Relevance in Geology and Astrophysics (R.F.8:13), 1978 Reference Year 


Share of Share of Share of Share of top 
Country relevant relevance top decile percentile 
papers(1) relevance relevance 
per cent i. 
UE CCC So UCLCS rae sates esi ereiel 60.0 62.8 6/64 Wed, 
ZENG L AM GME ene cverers es ele0 ee cee ec UES Teo HOSS 10.1 
(Ele aay epic Genin o camaccaus 6.4 4.7 3.0 a ieer4 
Vee Ta IS Waratere ener eter steyeters ote este os ao Le 36 (250) 
SEONESERGELMANY (sro cisces 0 056 Ded 4.0 be 0.6 
ALIS (ec ticleneratstete stele lctots starsle oe Die che De4 dine 
J Netherlands 26 seccsiss «ccc 2s sc ae) 24 Led oo 
BARBIE G oocytes DOKI NiSHE OOS Tey, alg 0.6 a 
AD cian oteteWelelotaleelole isislelsieieeie eis lol 0.6 RS -- 
(Il) SO elleleteh Ming oh os coma omoumOD Og Woy! 0.2 0.6 
AW emi © Peate tole c) ce ae ters 0 0s elas. eels Oe7 0.6 0.4 -- 
Var AW ESBS eid brat Gicrnoe ara ride 0.8 0.8 0.8 a 
1D WINES SIAINS Shen crac piereteteiats - 0.1 (ae 4 0.6 


(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate 
tapes, and those countries with at least 1% of fractionally allocated addresses are shown. 


TABLE 5. International Shares of Relevance in Physics of Materials (R.F.9:23), 1978 Reference Year 


Share of Share of Share of Share of top 
Country relevant relevance top) decile percentile 
papers(1) relevance relevance 
per cent 
(mee Ute dO SUAUES wires. sie sinks 01s Zin 24.0 24.35 Vee 
ye [Siaepletara! = Gt st SOAS Gaya Gor IRS) ices 1852 19-.6 
30 ASIST Olete oye te OA OO IO 11.4 950 8.0 1320 
i Wes. (emmeMmy occoasécc0co0506 DoW Dots D9 De Ti 
ELS cid sche Lol gslelsie.s, ottersisve.e1 670.2) e- 6's 7.0 6.1 Bio) 2.4 
Ga JAPAN) Poitevete secrsisye cles sale is 0 6.6 Sige Det 6.1 
Wie id 1 ecranece ale ciet states sie ecohecahe 7a) « Disa 3,09 4.0 eal 
OP LSWECEMN Meret asiete ake cies oterete sofa o's 3.0 Bey) Dee) 19.0 
OPP SWLUZEL LAN srcnecave's. « says steel oe 2.6 eal Zul Zu 
10 NetherlandS 2... ..ccscscucee 2.4 3.4 4.0 5.0 
ial SLI NCT Am ierevone etsbaveseels ate alelelcieleteie 2.3 ted 1.4 -- 
UI RC ANAC AG se secre se se ate mua mis ce <3 6.4 4.7 3.0 ke? 
13° AUSCT all Lar Bie wistcle sickeia osis'e oe 1.8 1.6 14 ae 


(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate 


tapes, and those countries with at least 1% of fractionally allocated addresses are shown. 
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TABLE 6. International Shares of Relevance in Applied Statistics, Population Biology (R.F.10:50), 1978 Reference Year 


Share Share Share Share Share Share Share Share 
of rel- of rel- of top of top of rel- of rel- of top of top 
evant evance decile per- evant evance decile per- 

Country pa- rel- centile Country pa- rel- centile 
pers(1) evance rel- pers(1) evance rel- 

evance evance 

per cent per cent 

1| Unated States: .... 38.4 45.0 49.9 oe i DES SWEGEMireratelelereleletiere 2.3 1.3 0.7 -- 

2 West Germany ..... Wee Pee) ile! 6.9 10) France: wsccn Ane 1.8 1.0 0.9 2.8 

3. (Bamadal \< aisssyain.eesla'e Ps6 8.5 Boo 7.0 11 Australia ..ccccse 1.8 267 3.3 3.4 

My jaqelletiel ooscooocco Hise. el 10.2 Diana (WZ UE awonodauond es 0.9 (BP) -- 

Die DADAM iaieisvelere sealers Sisk 1.8 0.5 -- ES. Waly: wisareisereye essere te2 0.5 0.3 -- 

6 AUStETLG sivscnccaes 4.6 3.0 2.7 Wrath 14 Netherlands ...... 12 G5 0.3 -- 

Hae SOc Vamdmejclartsleierals 2.6 1) 2.6 -- 15a) Denmarkierretera< jeleis tha’) Wee od 4.3 

8 Switzerland ...... ae 2.1 oe) -- 16 New Zealand ...... ent tec 0.4 -- 

(1) The addresses of relevant papers dated 1974, 1975 or 1976 were searched in the 74 to 76 corporate tapes, and those countries with at least 1% of 


fractionally allocated addresses are shown. 


TABLE 7. Canadian Institutions in the Top Decile of Relevance by Research Front, 1978 Reference Year 


See footnote(s) at end of table. 


Research Institution(1) Fre- Rank in Fre- Research Institution(1) Fre- Rank in Fre- 
front/ quency top per- quency front/ quency top per- quency 
rank in top centile in top rank in top centile in top 
decile per- decile per- 
centile centile 
Rokr Rebs 
1: 1: 1 University of Toronto ......... 40 2 4 2:10) University lof Alberta’ <\. <1 cn- 14 8 
2 McGill University ...c<ccescces 18 1 1 11 Queen's University at Kingston 10 - - 
3 University of British Columbia 13 - - 12 Dalhousie University ......... . 6 - - 
4 Sick Children's Hospital, co 13 University of Saskatchewan .... 3 - - 
TOLOMCO coveccvccccerecccsces 12 - - 14 Clarke Institute of Psychiatry, 
5 National Research Council, TOLONCO cevesccccsrcccscevecs 8 - - 
Ottawa ..... siele ose vsleese sae 12 3 2 15 Université Laval ...sesescecees 4 7 1 
6 Université de Montréal ........ 9 3 1 16 Children's Hospital, Montréal 5 - - 
7 University of Guelph .......... 7 1 17 Montreal Heart’ Institute ...... g) 10 3 
8 University of Manitoba ........ 11 - - 18 Université Laval Centre Hos- 
9 McMaster University .ccscscovee 2 - - opeciley (Meer sli Ghodaobacs 7 - - 
1G Université Laval wi sieicies sisneis-e's 8 - - 19 Ontario Cancer Institute, 
11. Ontario Cancer Institute, TOFONtO wascesseee sisi shiln ieyaiatte rate 2 6 
TOLONEO. cc veccccecccvccscvcce 6 6 1 20 Health Science Centre, Winnipeg 4 - - 
12 University of Western Ontario 6 - - 21 University of Western Ontario 4 - - 
13 Atomic Energy of Canada Ltd., 22 Kingston General Hospital ..... 2 - - 
Chalk River, Ontario .....0..5 5 5 23 Dept. of Agriculture, Ottawa .. 2 - - 
14 University of Alberta nse ew. 4 - 24 University of Calgary ......... 2 9 1 
15 Memorial University of 25 Université de Sherbrooke ...... 1 - - 
NeEwWFOURGLESMIG sie.6 «0 alenseimsyeiera 2 - 26 University Hospital, London, 
16 University of Saskatchewan .... > - CVE ARM) eo era re iersle wrslaieve/einrere's sel nye 3 - - 
17 York University .aseecsessaiecee 4 - 27 University of Waterloo ........ 1 - - 
18 National Research Council, 28 Hotel Dieu, Montréal ..... a sleee 1 - - 
SASKAUOOH! wisleieisleieiateielelee) ele/slelers 2 = 29 St. Joseph's Hospital, Hamilton 3 - - 
19 Royal Victoria Hospital, 30 Toronto General Hospital ...... 2 - - 
Montreal “Sissies sre e caste sslees 5) - 31 Memorial University of 
20 Queen's University at Kingston 2 - Newfoundland ....sceseseececes B - - 
21. Jewish General Hospital, 32 St. John's General Hospital ... 1 - - 
Montréal .ccvccccsscsvsccvece 2 = 33 Trent University ......... sfehevere 1 - - 
22 Université Laval Centre Hospi- 34 Queen Mary Vet. Hospital, 
Eel, QUEDECRETEY” vie. sista se ee 1 - Monmt real! sissies esses esis sees 1 - - 
23 Sherbrooke Medical Centre ..... 2 - 35 Windsor Clinic, Windsor ....... 1 - - 
24 Dalhousie University .........- 2 - 36 Hotel Dieu, Windsor ........ sles 1 - - 
25 Queen Mary Vet. Hospital, 37 Douglas Hospital, Montréal .... 1 - - 
Moni real! eieieisisteia love eieyolevera/s"siets)s 5 - 38 Canadian Red Cross, Toronto ... 1 - - 
26 University of Ottawa ...... year 1 - 39 WW Cross Cancer Institute, 
27 Ayerst Labs., Montréal ....see« 1 - EGMONEON .esvcccccves ain wretetel ers: i - - 
28 Dept. of National Defence, 40 Atomic Energy Canada Ltd., 
RE ONO) iy stsince ocaa (sais ie ars a /eieeneen 1 - PEndWwa, MANUEODA 6s visieciscisjeins 1 - - 
29 University of Calgary ..... Bago 1 - 41 Simon Fraser University ....... 1 - - 
30 Montreal General Hospital ..... 1 - 
31 University of Victoria, 
WVECEOLLA) 0,0 cies sjeisisinjelelae mleieisie'a 1 - 
32 Sunnybrook Hospital, Toronto 1 - 1: 5: 1 McGill University ........ ceees 7 1 1 
2 University of Toronto ......... 2B 4 2 
3 University of British Columbia 12 - - 
: 2: 1 University of British Columbia 572 1 > 4 University of Saskatchewan .... 7 3 1 
2 University of Manitoba ........ 4) 3 3 5 Queen's University at Kingston 9 2 4 
3 McGill University ....c.scccse 22 - 6 Sick Children's Hospital, 
4 University of Toronto ......+4. 31 - MOLONEO, we seccccccscevcvncece 12 - - 
5 Université de Montréal ........ 12 1 University of Manitoba ........ 6 - - 
6 Montreal General Hospital ..... 12 - 8 University of Ottawa .......+.. it - - 
7 McMaster University .......s00- 18 - 9 Royal Victoria Hospital, 
8 Sick Children's Hospital, Montréal ...cccocscoes ai a)eiete eons 9 - ~ 
MOLGHEDM.crslainin siatelene alalate ajelaistels 8 5) 10 McMaster University ...eceeeeee 3 - - 
9 Royal Victoria Hospital, 11 University of Western Ontario 3 - - 
Montréal .iciccie sie 00 sine woes 10 3 12 Université de Montréal ........ 2 - - 
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TABLE 7. Canadian Institutions in the Top Decile of Relevance by Research Front, 1978 Reference Year - Continued 


Research Institution(1) Fre- 
front/ quency 
rank in top 
decile 
Rane 
1: 5:13 Addiction Research Foundation, 
TOLOTIED wale acelin cs cede ale eS 3 
44 University of Calgary” .....5 5. z 
15 Ontario Cancer Institute, 
NGETONEG! ier. Clee elesisesicesecee 1 
16 University of Alberta ...... wise 2 
17 Ayerst Labs., Montréal ........ 1 
18 Université de Sherbrooke Centre 
HOS PRAT 5 scsive svc slaw ears wie 1 
19 Medical Research Council, 
Ottawa so... ee laieretetelersreteia. eve 5| 
20 Notre Dame Hospital, Montréal i 
21, \Wniversaty of WamdSor 6.166615. 1 
22 St. Boniface General Hospital, 
Winnipeg ....... leveislsieserstele tele rs 1 
23 Children's Hospital, Montréal 1 
24 Toronto Western Hospital ...... Z 
it of)? “ntversity cof Torento sistance. a7 
2 University of British Columbia 8 
3 elewilt, Winiveremey ssonogccgcace NZ 
4 “Université Laval so. .0.8 ses cee 5 
> “Unaversity of Alberta woes ese. 6 
6 Ontario Cancer Institute, 
MOTOMEO! Sees sccccccieve vcs a's 6 
7 Université Laval Centre Hos- 
pital, "QUEbEC LILY oeecewssu's 4 
8 National Research Council, 
ORT EWE) aye ntos ara stele eieie lays) aietevereie ets 4 
S “MeMaster University 2. ..c.asee« 2 
10 Queen's University at Kingston 7 
11. University of Western Ontario 3 
12 University of Waterloo .....%4% 2 
13 Ontario Cancer Foundation, 
MeSeOn soocdoosnodonosnoonc 2 
14 B.C. Cancer Institute, 
VANCOUVET scecccccesssecccce’s i 
15>) Dalhousie UAIVeErSity” <.ieisc sir. 5 
16 Université de Sherbrooke ...... 3 
i raversivey of Ottawa’ <i. si. <0 3 
18 Université de Montréal ........ 1 
19 Sick Children's Hospital, 
TOTOMEO won cccscccccveccccree 2 
20 University of Manitoba ........ 4 
21 Montreal General Hospital ..... 1 
22 Simon Fraser University ..... . 1 
23 Henderson General Hospital, 
Hamilton .... oielateletaieteverstar : 1 
24 Kingston General Hospital ..... 1 
25 University off Guelph) she... 4 
26 Manitoba Cancer Foundation, 
WAL EG are ere issternisareintets aisis tates 1 
We aie d “University of Toronto... ee. 16 
2 University of British Columbia i 
3 St. Luc Hospital, Montréal 20 
4 Université de Montréal ........ 9 
5 Toronto General Hospital ...... 6 
6 Sick Children's Hospital, 
TOPOMEG Sle cise cinslee ts scice eos e'e 8 
7 University of Alberta ......... 3 
8 McMaster University ..........- Ey, 
9 Dalhousie University .......... 4 
10 Queen's University at Kingston Bi 
11 MeGill University 22... .s..5s 3 
12 Montréal Heart Institute ...... 5 
13 WW Cross Cancer Institute, 
EGMOMEOM) o.6 ccicisc ciee occ eiele sale 2 
14 Shaughnessy DVA Hospital, 
Vancouver ..cecees sie cveescecs 1 
15 Kingston General Hospital ..... 1 
16 Notre Dame Hospital, Montréal 1 
17 St. Joseph's Hospital, Toronto 1 
18 Université Laval ...... oe ete 4 
19 Canadian Red Cross, Toronto ... 1 
20 Victoria Hospital, Halifax .... 1 
21 Royal Victoria Hospital, 
Montréal oi... 6 es SAU ORIN 1 
22 Dept. of National Defence, 
TOFOMEO wc ccccaccvacceevrctes 1 
23 Mt. Sinai Hospital, Toronto ... 1 
24 University of Guelph .......... 1 
25 Hotel Dieu, Kingston ........ an 1 
26 Toronto Western Hospital ...... 1 


See footnote(s) at end of table. 
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Institut ion(1) 


Fre- 
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Fre- 
quency 
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University of Toronto sy)... !6 
University of Guelph ....... eee 
McGill University 
University of Manitoba ........ 
University of Alberta 
University of British Columbia 
Royal Victoria Hospital, 

Montréal 
WneivermeEnys ILEWEN coongooaoogd06 
Toronto General Hospital 
Sick Children's Hospital, 

VOPOHCO ss crewiatere see ec eccccee 
MeMaster University .......c.s. 
Hotel Dieu, Kingston 
Memorial University of 

New fotima tama etevere te stele) clevelvieteiars 
University of Western Ontario 
Children's Hospital, Montréal 
University of Waterloo 
Queen's University at Kingston 
Wellesley Hospital, Toronto ... 
University Hospital of London 
Montréal General Hospital 
University of Windsor 
Hotel Dieu, Québec City 
University of Calgary 
Merck Frosst Labs., Montréal 
University of Alberta Hospital 
Queen Mary Vet. Hospital, 

Montréal 
St. Michael's Hospital, Toronto 
University of Saskatchewan .... 
Queen Mary Vet. Hospital, 

SE. dORM"S eiv.c ss 5 oietyelein ees 
Dalhousie University 
Camp Hill Hospital, Halifax ... 
Notre Dame Hospital, Montréal 
Children's Hospital, Winnipeg 
Health Science Centre, Winnipeg 
Halifax General Hospital 
Dept. of Agriculture, Saskatoon 
St. Justine Hospital, Montréal 
Mt. Sinai Hospital, Toronto ... 
Sunnybrook Hospital, Toronto .. 
Ottawa General Hospital 
Ottawa Civic Hospital 


eee e eee ene 


Université de Sherbrooke ...... 
University of Toronto 
Université Laval 
University of Alberta 
University of Waterloo ........ 
University of Calgary ......e.. 
Carleton University 
Sick Children's Hospital, 
TOTONEO coccscvees cc ccccvenes 
University of Western Ontario 
University of British Columbia 
Memorial University of 
Newfoundland 
York University 2. sccccsvecueee 
Concordia University 
Queen's University at Kingston 
Ontario Cancer Institute, 
Toronto .. 
Université Laval Centre Hos- 
pital, Québec City .......... 
Simon Fraser University 
McMaster University ....s.seee- 
University of Ottawa 
Jewish General Hospital, 
Montréal .cccscscccncccens tere! 
University of Guelph .......e. 
Dept. of Environment, Winnipeg 
Sudbury Algoma Sanatorium 
Laurentian University of 
Sudbury 


eee eee e reer eee eeee 


National Research Council, 

Ott awa 
University of British Columbia 
University of Waterloo 
University of Calgary ......... 
McGill University 
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TABLE 7. Canadian Institutions in the Top Decile of Relevance by Research Front, 1978 Reference Year - Continued 


Research Institut ion(1) Fre- Rank in Fre- Research Institution(1) 
front/ quency top per- quency front/ 
rank in top centile in top rank 
decile per- 
centile 
Rete R.F. 
2eq53 6 Université Laval erie. ccciee sm 3 - - 2:21:31 Dept. of Environment, Montréal 
7) University Of Toromto” <i... 3 - - SZ Brock UNIVEESICY ficsmediscwn noise 
8 Ecole Polytechnique, Montréal 6 - - 33 Sweltzer Creek Salmon Res. 
9 Simon Fraser University ....... 1 - - abi, Cultus Lakes Be. icine 
10 Health and Welfare Canada, 34 Dept. of Environment, Edmonton 
DEL AWA) 6 26. cine ele cis i010 «/eieie feuriers zi - = 35 Concordia University ....-.0.s. 
11 Dept. of Agriculture, 36 Université de Montréal ........ 
HeGhibiraldges teretoreroclerlelotietatere ee 1 - - 37 Dept. of Environment, 
12 University of Western Ontario 1 - - Bia Teli ooosacavconadonoo= 
13 Pulp and Paper Research i 
Institute, Montréal ......... 1 - - 
2:24: 1 National Research Council, 
ZOG W  Wieleatilil Winitienenhay, oasqnodqnogn6 10 - - Ottawa ccc ccc nec cc ce cies evince 
Z University of Guelph” ss. <j. cree 9 - - ZPURIVEESLEQ OF TORONES wc occ 
3 Sick Children's Hospital, 3 Sick Children's Hospital, 
TOLONCO wecceeseecseoee vecese 8 2 1 TOLTONEO sivccescccccecs sis s)0\sicle 
4 University of British Columbia 5 1 1 4 Memorial University of 
5 Health and Welfare Canada, Newfoundland) .. c<siscneswess vic 
DEE AWA. « sicieicjeneas ons 'ei0 Sisiein ors 4 - - > York University seh cise n se nines 
6 McMaster University .........0 5 - - 6 McGill University ...... se eeeee 
iM SUREMOWM ye TOMONTG! forces este olaretsn nest 1 3 1 7 Ayerst Labs., Montréal ........ 
8 University of Toronto ...... . 4 = - 8 University of Manitoba ........ 
9 Dept. of Agriculture, Ottawa 5 - - 9 Health Science Centre, Winnipeg 
10 University of Alberta ........ 5 53 - = 10 University of Western Ontario 
11 University of Saskatchewan .... 2 - - 11 University of British Columbia 
12 Royal Victoria Hospital, 12. Dr. CA Janeway Child Health 
Neiaeecel! osechocenpodns sacs ae z - - Centres St. Johms se tewiswel o's 
13 York URiversity ceecccessvncces 1 - - 13 University of GWelph ...ccsece 
14 University of Manitoba ........ 1 - - 14 McMaster University ....ceseoee 
15 Health Science Centre, Winnipeg 1 - - 15 Université Laval Centre Hos- 
16 Montréal Heart Institute ...... 4 - - pital) Québec City tere syrclerie 
17 Dept. of Agriculture, 16 Enfant Jesus Hospital, 
Lennoxville, Québec ......... 1 = e Québec City .6ciesccsnccseves 
18 St. Justine Hospital, Montréal A - - 17 Dept. of Agriculture, Ottawa .. 
19 Université de Montréal ........ 1 - - 18 National Research Council, 
20 Jewish General Hospital, SASKALOOM! siclefelereeietereleisrosere eee 
MOMER Ga actetersteletulotary/afetctetetslot ets 1 - - 19 Royal Victoria Hospital, 
21 Ottawa General Hospital ....... 1 - - Momtreal ss. ciel eas seevcces 
22 University of Ottawa ....cscves 1 = = 
23 Lachine General Hospital, 
ileinioetiall Soanooconquponccsoods 1 - - 
24 Children's Hospital, Montréal 1 - - 2:26: 1 University of Toronto ......... 
2 Clarke Institute of Psychia- 
2:21: 1 University of British Columbia ae - - LEY, TOPONCO sccecccsnccs acces 
2 Dept. of Environment, Halifax 12 - - 3 Queen's University at Kingston 
3 Dalhousie University ......+... 6 - - 4 St. Francis Xavier University, 
4 National Research Council, AME RGONLSH peetecc w eles ele se fnpark 
Saskatoon oeccccccccsseovvcee 8 1 7 5 York University cecscccscsecene 
5 Dept. of Environment, Vancouver 43 - - 6 University of British Columbia 
6 University of Toronto ...cceeee 8 - - 7 Royal Victoria Hospital, 
7 Dept. of Agriculture, MOmErEe ale ates sic sicine c slelsieieisisis 
Summerland, B.C. sseccseeseee 6 - - 8 Concordia University ...ceceees 
8) Meta Waiver Sie y® ceivisw.e.ciels «iniriete 2 2 1 9 St. Jean de Dieu, Montréal .... 
9 University of Alberta ......e0. 5 - - 10 University of Waterloo ........ 
10 Dept. of Environment, 11 University of Manitoba ........ 
FEECELICEON oc cece cesiccscccns 3 - = 
11 University of Manitoba ........ >) - - 
12 Simon Fraser University ....... 3 5 Z 
13. B.C. Research Council, 2:27: 1 McMaster University ...eececres 
VANCOUVET cesecvescvesecsvors z - ~ 2 Chedoke Hospital, Hamilton .... 
14 University of Ottawa ........0 2 - - 3 Memorial University of 
1S YORK URUVEL SUG wetsioielslerelelaleisiele sie 1 - - Newfoundland) s.. 6 0.c ce ccc coe 
16 Atomic Energy of Canada Ltd., 4 Queen Mary Vet. Hospital, 
Pinawa, Manitoba ......seseeee 4 - - Momba all War cravelstsinielstslstcheleisielerclols 
17 University of Waterloo ........ 3 - - 5 McGill University ....seceeseee 
18 Perkin Elmer Cdn. Ltd., 6 University of Western Ontario 
MGIC Lars lola) oletetelotetolsisialstetelsters i] 33 1 7 University of Alberta .......0. 
19 Dept. of Environment, 8 Calgary General Hospital ...... 
Sault St. Marie ccceccsccccce 1 9 University of Ottawa .......eee 
20 University of Guelph .......... 2 - 10 Children's Hospital, Montréal 
21 Université du Québec .......... 1 - - 11 Wellesley Hospital, Toronto ... 
22 Dept. of Agriculture, Saskatoon 1 - - 12 Kingston General Hospital ..... 
23 Dept. of Agriculture, London .. 1 - - 13 Queen's University at Kingston 
24 Memorial University of 14 Ottawa General Hospital ....... 
Newfoundland! svis os c's ciiciesiniatvis 1 - - 15 Toronto Western Hospital ...... 
25 University of Western Ontario 1 - - 16 University of Toronto ........-. 
26 Dept. of Environment, Hamilton 1 - - 17 University of Alberta Hospital 
27 Dept. of Environment, Winnipeg 2 - - 
28 Fisheries Research Board, 
HeLa axis sie slsterele'siele(crsieleislstelsters 1 - = 
29 Ont. Ministry of Agriculture & 2:32: 1 University of Guelph .......... 
Food, Guelph, Ontario ....... 1 7 - 2 Université de Montréal ........ 
30 Ont. Ministry of National 3 University of Saskatchewan .... 
Resources, Delhi, Ontario ... 1 - - 4 Dept. of Agriculture ....eeeees 
5 UMKMOWN acevceccvcvsccscrevccee 


See footnote(s) at end of table. 
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TABLE 7. Canadian Institutions in the Top Decile of Relevance by Research Front, 1978 Reference Year - Continued 


Research 
front/ 
rank 


nNUPWY 


Zoos 


PououmItnvew 


is 


eae 4 


Wry 


2:46: 


Institut ion(1) 


Memorial University of 

Newfoundamd s.cccccccees 
University of Western Ontario 
University of Ottawa ... 
University of Alberta ......... 
University of Manitoba ... 
Atomic Energy of Camada Ltd., 

Pinawa, Manitoba ............ 
Dalhousie University 
National Research Council, 

Deere GnoucnuontodpandaoogGK 
Dept. of Agriculture, 

Sault Ste. Marie! we. cisin «see we 
Dept. of Environment, Vancouver 
B.C. Cancer Institute, 

Vancouver 
University of British Columbia 
BrockaUmiVenSiey” “yeleiscreisis eis\sisials 
University of TOronto ...ccesss 
Ontario Cancer Institute, 

Toronto® ... 
UNniVierisaty loi GUellphy sis jeteveis ex 0ie 
eeRUUL Winiwersrtey ssoooooduc5c 
WW Cross Cancer Institute, 

EGMOMEGR sie 606 0i9 0010's seeeee 
Halifax Victoria General 

O'S Pils GUL vous tokens oie) siete aie ohe atelier 
UnaVersitymof Caligarye i). cies. 
Sick Children's Hospital, 


sees 


eee eeee 


ee eee 


Ce 


seer reer rere sone 


VOUISEY OF We,cio) erelete eiaislasieio'e 10 \s\e fon 
Princess Margaret Hospital, 
TOBOMEOM exe ore late hgoDCOCOdad eye 


St. Michael's Hospital, Toronto 


University of Toronto ......... 
University of Guelph 
University of British Columbia 
MeGill University ..seccssscees 
University of Ottawa ......e.e. 
Dept. of Environment, 

SES IOMMUSI to is evelatelelelete e wiele ovslole 
Université de Montréal ........ 
University of Calgary 
Université de Sherbrooke ...... 
Dalhousie University ...cce5ee. 
Université Laval Centre Hos- 


se eeeecees 


PHualls QUEDIES ELDYI ei slsleisiereore 
Health and Welfare Canada, 
OE RAWAM testers lorieleysy stele steile tompirelisi 


National Research Council, 
Halifax 
Dept. of Agriculture, Ottawa 


Bere creer reressesecce 


University of Alberta ......... 


McMaster University .-.ecssosee 
Flin Flon General Hospital, 

Manitoba ..... ponoogoondd 
University of British Columbia 
University of Toronto 
University of Guelph ...... tees 
University of Alberta ......... 
Toronto General Hospital ...... 
Ayerst Labs., Montréal ........ 
Vancouver General Hospital .... 
McGill University ............. 
Douglas Hospital, Montréal .... 


oes 


ee eeecons 


Sick Children's Hospital, 
TOLONEO cesccsscerveccciscsic eee 
University of Toronto ......... 
McGill University s...ss.6. eo. 
Royal Victoria Hospital, 
Montréal) sites. 
University of Manitoba ........ 


eee mere een ernne 


McGill University ...eccceecees 
University of Toronto 
University of Western Ontario 
Ontario Cancer Institute, 
Toronto ... 
University of Manitoba ........ 


seers eoee 


Seer eee eer e sess ese 


See footnote(s) at end of table. 


Fre- Rank in Fre- Research Institution(1) Fre- 
quency top per- quency front/ quency 
in top centile in top rank in top 
decile per- decile 
centile 
RB. 
ASS (5 Winey oii (elle? Soonqnand 2 
8 = - 7 University of British Columbia 2 
2 1 1 By WUniversmte cup QUEDEG) Kiereisieicsiein 1 
3 - - 9 Wellesley Hospital, Toronto 1 
6 - - 10 Dept. of Agriculture, Vancouver 1 
3 = = 
2 - - 2:56: 1 University of British Columbia ee 
7 - - 2 Montréal General Hospital ..... 2 
Oy Me Gialen imation Sate yitave rs tetetelsisisisiereie! 4 
6 - = 4 Ontario Research Foundation, 
TOPOMEG ac a iieierels ces soa die.e ercisie Zz 
1 - - 5 Wellesley Hospital, Toronto ... 2 
1 - - 6 Sick Children's Hospital, 
MOTROMED) "ofereretayers roio/s: <lellsreis) si cee 2 
1 - - 7 Health Science Centre, Winnipeg 1 
- - 8 Queen Mary Vet. Hospital, 
1 - - Molgumcewl Goaogodsoagoc evoisteteys 5 1 
5} - - 9 Atomic Energy of Canada Ltd., 
Chalk River, Ontario <jcseuiee 1 
1 - - 10 University of Western Ontario 2 
1 - - (i) iW Yoir llega Godgacnca 1 
- - 12 Foothills Hospital, Calgary 5 
13 Université de Sherbrooke Centre 
1 - - THES po tea esyevatiereraretateveceshelstaissecniis 1 
14 McMaster University ..e.seceses 1 
1 = = 15 Montreal Neurological Hospital 1 
1 = = 
1 - - 3: 4: 1 Atomic Energy of Canada Ltd., 
Chalk River, OnNGAELO! scenes 9 
1 - - 2 University of British Columbia 7 
it - - 3 McMaster University ....sscsecs 9 
4 Université de Montréal ...scees a 
5 McGill University ......-... oes B 
7 - - 6 University of Manitoba ........ 4 
5 - - 7 Universaté Laval ..... a2 ole oie ieisls 4 
2 - - 8 University of Toronto .....cc0e Az 
2 1 2 9 Queen's University at Kingston 5 
6 - - 10 National Research Council, 
DEE AW erate oi aiere nie mn nl aleceolaieceisiniele: 4 
2 - - 11 University of Guelph ......s06 3) 
2 - - 12 University of ObLaWa ..ccewisicse 1 
1 - = 13 Simon Fraser University ......- 2 
il - = 14 University of Western Ontario 4 
2 - - 15 Carleton University ....sseoses 3 
16 University of Alberta ......... 1 
3 - - 17. Xerox Cdn., Toronto seccscceses 1 
1 = = 
3: 9: 1 National Research Council, 
1 - - BEE AWA: 6c sis ciaieia sine ele oes es vce 9. 
1 - - 2 University of Toremtea .. 2.5 sce 20 
3 University of Alberta ......... 9 
4 University of British Columbia 13 
1 - - 5 University of Guelph .....cesce 2 
6 University of New Brunswick ... 3 
7 Université de Montréal ........ 5 
| - - 8 McMaster University ...... os Z 
9 Université Laval ...... an8)'9/ 616 'eu8i 3 
2 - - 10 University of Western Ontario 3 
2 - - 11 University of Windsor ....... a4 d 
2 - - 12 University of Manitoba ........ 1 
1 - - 13 Université de Sherbrooke ...... 2 
4 - - 14 Simon Fraser University ..... . a 
5 - - 15 York University ....csccccce wale it 
2 a = 
1 ~ - 
2 = = 
1 - - 3:10: 1 University of Toronto ......... 19 
2 National Research Council, 
Ottawa ...ssccoes eve ceen cence 14 
3 Atomic Energy of Canada Ltd., 
4 - - Chalk River, Ontario ....... 9 
3 - - 4 University of Windsor ......... 3 
1 - - 5 University of British Columbia 14 
6 University of Saskatchewan .... 3 
1 - ~ 7 Université Laval .....eeeeeee oe 9 
1 - . 8 Queen's University at Kingston 5 
9 McMaster University ..ccceccees 14 
10 Trent University .eccecccesceae 2 
S) 1 1 11 University of Western Ontario 8 
6 - - 12 University of Alberta ......... 4 
2 - - 13. University of Waterloo .......- 11 
14 Université de Montréal ........ 9 
3 - - 15 University of Manitoba ........ 3 
1 Z 1 16 McGill University ...ecceeeeeee z 
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TABLE 7. Canadian Institutions in the Top Decile of Relevance by Research Front, 1978 Reference Year - Continued 


Research Institution(1) Fre- Rank in Fre- Research Institution(1) Fre- Rank in Fre- 

front/ quency top per- quency front/ quency top per- quency 

rank in top centile in top rank in top centile in top 
decile per- decile per- 
centile centile 

R.F. R.F. 

3210317 Dalhouste UNnIversaty  iccrweiewerre 1 - - 4: 3:25 Pulp and Paper Research 
18 Barringer Research Ltd., Instatute;, Montréal <2... si. 1 - - 

TO BOMUEG te lorolet slelelate le c¥eisi sts leteteelers 1 - - 26, CdnacnPackerssitd. TOrOnton cite. i - - 
19 University of Ottawa <2. .c..cce 1 12 1 27 Université de Moncton ......... i - - 
20 Université de Moncton ..... sisiete 1 - - 
24 York University sis w0s nie #is.0/a)a.ts 3 - - 
22 Bowmar Cdn. Ltd., Ottawa ...... 1 - - Stites Unaiversaty of Calgary) vince. 21 - - 
23 Linear Tech. Inc., Hamilton ... 1 - - 2 University of Toronto: asasicwsies 7 - - 
24 Simon Fraser University ....... 1 - - 3 National Research Council, 
25 FMC Cdn. Ltd., Squamish, B.C. 1 - - See oooosscnsosconnoccs 7 - - 
26 Université du Québec .......... 1 - - 4 University of Western Ontario 3 - - 
27 Dept. of National Defence, 5 University of Guelph 2... sense. 5 - - 
QS ECHO UUY Melelefeteteleleintalereleleterets 1 - - 6 McMaster University ......c.e0~ 3 - - 
28 Unican Electrochem. Prod., 7 Queen's University at Kingston 3 1 1 
Momime all iaresc\ererateielsielocnneie acces 1 - - 8 University of Alberta) ..ce.s0e f - - 
29 Xerox Edn., TOromto .essicaccnse 4 - - 9 Simon Fraser University ....0+. i - - 
10 Dept. of Agriculture, Ottawa .. 2 - - 
11 Dept. of Environment, 
3:14: 1 Dept. of Environment, FREGErUCEOM | oie <icisle e.c1n:ninle steven. 1 - - 
Wollivadilios INicSia1 \eievere s/oleleleleielere 20 1 6 12 Dept. of Agriculture, London, 
2 Dalhousie University .......... 7 - - OnE AE TO! eyets aoe ina wise ae teinlaieisiaiare zZ - - 
3 Dept. of Agriculture, Ottawa .. 2 - - 13 Carleton University ....ccccsecs 1 - - 
4 Health and Welfare Canada, 14 University of British Columbia 1 - - 
DECaWa! Be siere alelelsieiors © oe cecccee 6 1 15 University of New Brunswick ... 1 - - 
5 University of Waterloo ........ 10 - 16 University of Saskatchewan .... 1 - - 
6 University of Alberta ......... 5 - - 17 Dept. of Agriculture, Windsor 1 - - 
7 Dept. of Environment, Halifax 5 - - 
8 Université de Moncton ......... 6 - - 
Oo MeGill Umiversity: access «0 os 3 - - 
10 University of Toronto ......... 5 - - 5:20: 1 National Research Council, 
11 University of Manitoba ........ 2 2 Zz DEtaWa <cccccncccceveve elsvele sin u 3 1 
12 Gulf Oil Canada Ltd., Toronto 1 - - 2 University of Guelph .......ee. 8 2 Zz 
13 Alberta Research Council, 3 University of British Columbia 5 - - 
Edmonton ...... eee cerecceceee 1 - - 4 University of Waterloo ........ 2 il 2 
14 National Research Council, 5 Concordia University ....cccess 2) - - 
Ottawa ..cccsoes 80 ec wien e aie 1 - - 6 University of Western Ontario 6 - - 
15 Dept. of Agriculture, Winnipeg 1 - - 7 University of Manitoba ........ 2 4 2 
16 Dept. of Environment, 8 University of Alberta ...ceesee 2 - 
Vancouver ..... Bile) sis e\6/0 seccee 1 - - 9 University of Saskatchewan .... 2 - - 
17 Dept. of Environment, 10 University of Toronto ..... eee 2 5 2 
Sault, Stes) Marte. i aieucieiwisteisre Z - - 11 University of Calgary ....... ate 3 - - 
18 Ont. Ministry of Environment, 12 Dept. of National Defence, 
TODOMED: cic siete eee cic al sie oreietavere 3 - - Ottawa ..ccsccccccescces eecee 1 - - 
19 Energy, Mines and Resources, 13. Ayerst Labs., Montréal ........ 1 - - 
Ottawa’ <siee cisccisieceese ee eeeee zZ - - 14 Queen's University at Kingston 1 - = 
20 University of Saskatchewan .... 3 - - 15 Xerox Cdns,, forontel....ccee wea. 1 - - 
21 Memorial University of 
Newfoundland ..ccovsccccccves 4 - - 
22 Dept. of Environment, Winnipeg 2 - - 53253 1 Carleton University 2 ose seesi 1 - - 
23 Umiversity Hospital of 2 University of Guelph .......... 1 - - 
Saskatchewan, Saskatoon ..... 3 - - 3 McGill University ........+. ees 2 - - 
24 Carleton University ....eeseeee 3 - - 4 Queen's University at Kingston 1 - - 
25 Health and Welfare Canada, 5 Université de Sherbrooke ...... 1 - - 
Halldfax: civieiccisic.c'c 101s cicicie eee 1 - - 
26 Ayerst McKenna & Harrison, 
MOMED EU We reieteistoleretsieisislefelelelsiers . 1 - - 5:29: 1 Dept. of Agriculture, 
‘ LONdON wees aio saielevsiarerereiseysle\s 10 - - 
2 Dept. of Agriculture, 

43 33 1 University of Alberta «sic. srecieie 39 1 ) Charlottetown ....++.e. oecves 8 - - 
2 University of Toronto ......... 34 - - 3 University of Guelph ........0.% 6 - - 
3 University of Western Ontario 20 Z 5 4 University of Alberta ......... Z Z 1 
4 University of Ottawa .......... 15 - - 5 University of Saskatchewan .... > - - 
5 University of Waterloo ........ 10 - - 6 Brock UMIVEFSILY 2 cisci.s ciewiee sien {I 1 1 
6 Queen's University at Kingston 11 - - 7 Dept. of Agriculture, 

7 McGill University ...cccecvscee 14 - - Lennoxville, Québec .......+. 1 
8 University of British Columbia 12 - - 8 Dept. of Agriculture, 
9 University of Gueliphiy 6006 ccs: 10 - - FREGEPLECOM sic-eis cele ate oieieteprinios 1 - - 
10 McMaster University a.csecssevs 4 3 2 9 National Research Council, 
11. University of Windsor ......... 6 - - SaskatOOn .sccssiecrece . 5 4 - - 
12 University of Manitoba ........ 2 4 1 10 Dept. of Agriculture, 
13 Simon Fraser University ....... 8 - = Swift Current, Saskatchewan 1 - - 
14 National Research Council, 11 York University .e.escoee eoecee Zz - = 
DEC AWA cocccccsccvcccccceses 5 - = 12 Dept. of Agriculture, 
15 Dalhousie University ....... stare 2 - - Wolfville, NsSs cies. eoiceewae 1 - - 
16 University of Saskatchewan .... 5 - - 13° Dept. of Agriculture, Winnipeg 1 - = 
17 Université de Montréal .....0.. 2 - - 
18 Bristol Labs. Cdn., Montréal 2 - - 
19 Dept. of Environment , 6:17: 1 University of British Columbia 1 1 1 
FRCAEHECE OM) yareicreterisieralorersietetstors 3 - - 2 Dalhousie University ....... aan 1 - - 
20 Université de Sherbrooke ...... 2 - = 3 University of Toronto ......... 1 - - 
21 York UMAVErSILY 0006s ccucciecace 3 - = 
22 University of Calgary ......+. . 3 - - 
23 Concordia University .....sseee 1 - - 6:30: 1 McGill University .....sccccvee Z - - 
24 Lakehead University, Energy, Mines and Resources, 
Thunder Bay. csjess's s:c/en0\ej0,0:0,0.0 1 - - DEEAWE aivieecvecsitwwsessriesien 2 - - 


See footnote(s) at end of table. 
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TABLE 7. Canadian Institutions in the Top Decile of Relevance by Research Front, 1978 Reference Year - Continued 


Research Institution(1) 
front/ 
rank 
Rabie 
6:30: 3 Atomic Energy of Canada Ltd., 
Chalk River, Ontario ........ 
4 Dept. of Environment, 
VANCOUVER) “sieve weisieisie es weveisterere 
Si578 University of Toronto csaceices 
2 National Research Council, 
Ottawa ..... ees eeeesccecs ars 
5 Palhousie University esc siviesic 
4 University of British Columbia 
5 Dept. of National Defence, 
TOLONEO cee ececcsccsceccscees 
7:34: 1 National Research Council, 
OOrEWE, cooocan Boon oslo 
Ze Una VeErSIty oF OLCAWE srcleciercterenats 
3 University of British Columbia 
& (University of Waterloo “1... 6. 
>» UniVversitysof \Alibertal iyt1.1-<t.1 
6 Université de Sherbrooke ...... 
7 Dept. of Environment, Ottawa 
8 Nova Scotia Technical College, 
Hall aiiaxe Grete sictsreretsy sie stoic ol 53000 
S) WWeevUk (Umer Gooopogcud6Hn 
10 Queen's University at Kingston 
41 Université de Montréal ....... 5 
12 McMaster University .......ss... 
fom Universit és Pavia src. sels e/l< eee teters 
14 University of Calgary ......... 
15 Synerude Cdn. Ltd., Edmonton .. 
Neais ee tamversity iat Obbawal cnc sperm 
2 University of Toronto ......... 
3 National Research Council, 
Ottawa ....... ee cccece eeecees 
4 University of Alberta ......... 
DRAUMAVETES LE VROtn Water LOO! raereretatctere 
6 Atomic Energy of Canada Ltd., 
Chalk River, Ontario ........ 
7 University of Guelph .......... 
8 Royal Military College, 
KainGStGial s.c.5 sivyeinie:5.0 dteleva) aheterais| 
9 Brock University ....-e.sscsees . 
7:51: 1 University of Western Ontario 
2 MeMaster University sie. saes oss 
7:53: 1 Inst. Rec. Hydro Québec, 
Momtireiall! %o cic siecle. 0/sie)visieleie oe 
2 University of Alberta ....... ave 
3 University of Western Ontario 
4 University of Windsor ......... 
5 National Research Council, 
DEC aWa cececcsecsevrerseoseses 
6 University of British Columbia 
7:79: 1 University of New Brunswick 
74 Well WAIVE Googonconcoud 
3 McMaster University ...sescceee 
4.) Norandal, Momtmeall "si. sssjeretetele sere 
Bsi3z7 7 University of Toronto 2.2.0. 
2 Energy, Mines and Resources, 
VASE Aitetelsjetelsveraste whalete: eheleracaie 
3 Energy, Mines and Resources, 
OL LaWa cacccccccecescsececcves 
4 Memorial University of New- 
foundland ....... Daveaecicisee ° 
5 University of Western Ontario 
6 University of Alberta .....+. ei 
7 National Research Council, 
OL taWa ceeceecccscccccccesecs 
8 Queen's University at Kingston 
9 University of British Columbia 
10 Université de Montréal ..... oes 
11 Université Laval ........ aisle ese 
12 University of Calgary ......... 
13. York University ceesceeceeee eee 


See footnote(s) at end of table. 
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R.F. 


9:48: 


9:81; 


10:38: 


(gS 


NOSa9is 


Toh ek 


DUE WwW 
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Institut ion(1) 


Université Laval ........ 
National Research Council, 
Ottawa 
University of Toronto 
Université du Québec .. 
Unviersity of Waterloo 
Concordia WUnRVerSAty Wieesls.\se'= . 


se eeee 


see eeeoe 


University of Saskatchewan .... 
University of Alberta ......... 
National Research Council, 

Ottawa 
University of British Columbia 
Dept. of Communication, Ottawa 


Queen's University at Kingston 
University of Alberta 
Dept. of Environment, Halifax 
Dept. of Environment, Hamilton 
Dept. of Agriculture, Ottawa 
Acadia University, 

Wolfville, N.S. 
National Research Council, 

PEW AWa tejcrejnie. ute wiee\nsetnieretslnietelesie 
Dept. of Agriculture, 

Wolfville, N.S. 


Dept. of Environment, Toronto 
McGill University 
Dept. of Environment, Montréal 
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88-501E An Indicator of Excellence in Canadian Science 
88-502E International Payments and Receipts for Technology 
88-503E Technology and Commodity Trade 
88-504E Patents as Indicators of Invention 
88-505E Industrial Productivity and Research and Development Indicators 
88-506E A Framework for Measuring Research and Development Expenditures in Canada 
88-507E An Indicator of Excellence in Canadian Science: Summary Report 
88-508E Human Resources for Science and Technology (forthcoming) 
These publications are also available in French. 


Statistical reports for most indicator series are being developed for annual publication by 
Statistics Canada. Existing publications are listed below. 


STATISTICAL PUBLICATIONS 


Cat aloque 

88-001 Science Statistics, Monthly 

88-201 Science and Technology Indicators, Annual 

88-202 Industrial Research and Development Statistics, Annual 
88-203 Resources for Research and Development in Canada, Annual 
88-204E Federal Scientific Activities, Annual 


88-205E Industrial Research and Development Facilities, Annual 
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